Higher Order Derivatives

that is, interchanging the order of the partial derivative leads to the same answer. This is not accidental as the following proposition shows, the proof of which will be omitted.

🔗

Proposition 4.38. Symmetry of second partial derivatives.

Suppose that

f,

defined on

D \subset R^{N}

has continuous second order partial derivatives. Then

🔗

\frac{\partial^{2} f}{\partial x_{j} \partial x_{i}} = \frac{\partial^{2} f}{\partial x_{i} \partial x_{j}}

for all

i, j = 1, \dots, N .

🔗

Note 4.39.

Examples show that the assumption that the partial derivatives be continuous is essential for the above result to be true!

🔗

As a natural generalisation of partial derivatives we studied directional derivatives. We now want to look at higher order directional derivatives. Given a function

f

defined on

D \subset R^{N}

and a unit vector

v = (v_{1}, \dots, v_{N})

we set

🔗

\begin{aligned} \frac{\partial^{2}}{\partial v^{2}} f (x) & := \frac{\partial}{\partial v} (\frac{\partial f}{\partial v}) (x) \\ \frac{\partial^{3}}{\partial v^{3}} f (x) & := \frac{\partial}{\partial v} (\frac{\partial}{\partial v} (\frac{\partial f}{\partial v})) (x) \end{aligned}

etc.

🔗

In Proposition 4.28 we derived a formula for the directional derivative. We found that

🔗

\frac{\partial f}{\partial v} (x) = (grad f (x)) \cdot v

grad f

is continuous at

x .

To compute the second directional derivative we can apply the same formula to the function

(grad f (x)) \cdot v .

Doing that we get

🔗

\frac{\partial^{2}}{\partial v^{2}} f (x) = grad ((grad f (x)) \cdot v) \cdot v .

🔗

To derive a more explicit formula for the above expression we compute the partial derivatives of

(grad f (x)) \cdot v :

🔗

\begin{aligned} \frac{\partial}{\partial x_{i}} (grad f (x)) \cdot v & = \frac{\partial}{\partial x_{i}} \sum_{j = 1}^{N} \frac{\partial f}{\partial x_{j}} (x) v_{j} \\ = \sum_{j = 1}^{N} \frac{\partial^{2} f}{\partial x_{i} \partial x_{j}} (x) v_{j} . \end{aligned}

🔗

Therefore,

🔗

\begin{aligned} \frac{\partial^{2}}{\partial v^{2}} f (x) & = grad ((grad f (x)) \cdot v) \cdot v \\ = \sum_{i = 1}^{N} \sum_{j = 1}^{N} \frac{\partial^{2} f}{\partial x_{i} \partial x_{j}} (x) v_{i} v_{j} . \end{aligned}

🔗

If we set

🔗

\begin{matrix} (4.5) & H_{f} (x) := [\begin{matrix} \frac{\partial^{2}}{\partial x_{1}^{2}} f (x) & \dots & \frac{\partial^{2}}{\partial x_{1} \partial x_{n}} f (x) \\ ⋮ & ⋱ & ⋮ \\ \frac{\partial^{2}}{\partial x_{n} \partial x_{1}} f (x) & \dots & \frac{\partial^{2}}{\partial x_{n}^{2}} f (x) \end{matrix}], \end{matrix}

and

🔗

v = [\begin{matrix} v_{1} \\ ⋮ \\ v_{N} \end{matrix}] and v^{T} = [\begin{matrix} v_{1} & \dots & v_{N} \end{matrix}],

then, using matrix multiplications, we can rewrite the second directional derivative by

🔗

\frac{\partial^{2}}{\partial v^{2}} f (x) = v^{T} H_{f} (x) v .

🔗

This motivates the following definition.

🔗

Definition 4.40. Hessian matrix.

The matrix

H_{f} (x)

given by (4.5) is called the Hessian matrix of

f

x .

🔗

Remark 4.41.

It follows from Proposition 4.38 that the Hessian matrix

H_{f} (x)

is symmetric if

grad f

is continuous at

x .

🔗

Example 4.42.

The Hessian matrix of the function in Example 4.37 is

🔗

H_{f} (x, y) = [\begin{matrix} 6 x - 6 y & - 6 x \\ - 6 x & 0 \end{matrix}] .

🔗

Note that the matrix is symmetric.

🔗

Let us summarise what we just found.

🔗

Proposition 4.43.

Suppose that

v

is a unit vector, and that

f

has continuous first and second order partial derivatives at

x .

Then the second directional derivative in the direction of

v

is given by

🔗

\frac{\partial^{2}}{\partial v^{2}} f (x) = v^{T} H_{f} (x) v,

where

H_{f} (x)

is the Hessian matrix of

f

x

defined by (4.5).

🔗

In principle we could continue to apply (4.3) to compute the third, fourth and higher directional derivatives. However, for later purposes we only need the second derivative. We next want to use what we learnt to find `Taylor polynomials’ for functions of several variables.

🔗

Prev Top Next