Vector Calculus: An Introduction

Using the above facts we want to derive conditions for maxima and minima of functions of several variables. To do so assume that $f$ is defined on a domain $D$ in ${\mathbb{R}}^N\text{.}$ Suppose that $f$ attains a (local) maximum (or minimum) at some point $\mathit{a}=(a_1,\dots ,a_N)\text{.}$ For every $i=1,\dots ,N$ we can then define a function of one variable, $t\text{,}$ by setting

The graph of this function is a cross-section of the graph of $f$ in the $i$-th coordinate direction as seen in Figure 4.1 for a function of two variables. If $f$ attains a maximum (or minimum) at $\mathit{a}$ then $g_i$ attains a (local) maximum (or minimum) at $t=0$ for all $i=1,\dots ,N\text{.}$ If $f$ has partial derivatives in all directions then by the criterion for single variable functions and the definition of the partial derivatives we have

for all $i=1,\dots ,N\text{.}$ We therefore have the following theorem.

As with the first derivative test for maxima and minima we can look at cross sections of the graph of the function $f$ in the coordinate direction. If $\mathrm{grad}f(\mathit{a})=0\text{,}$ then

implies that the cross sections in the coordinate directions have a maximum at $\mathit{a}\text{.}$ The question is whether this guarantees a maximum at $f(\mathit{a})\text{.}$ Unfortunately, the answer is NO! The reason is that we only look at cross-sections of the graph in the coordinate directions. The condition only tells us that the function attains a maximum along these cross-section. In other directions it may be decreasing. As an example look at the function $f(x,y)=4x^2{y}^2-x^4-y^4\text{,}$ whose graph is displayed in Figure 4.49. Along the cross-sections parallel to the axes through $(0,0)$ the function attains a maximum, but along the cross-sections in the diagonal directions it attains a minimum. You can easily verify that formally.

Hence we have to make sure that we have a maximum along every cross-section. This means that all first directional derivatives must be zero, and all second directional derivatives of $f$ at $\mathit{a}$ must be negative in order to ensure a maximum. Hence let $\mathit{v}=(v_1,\dots ,v_n)$ be an arbitrary unit vector, and define

Assume that all first and second order partial derivatives of $f$ are continuous at $\mathit{a}\text{.}$ According to the previous reasoning $f$ attains a maximum at $\mathit{a}$ if $g_{\mathit{v}}^{\prime }(0)=0$ and $g_{\mathit{v}}^{″}(0)<0$ for every unit vector $\mathit{v}\text{.}$ By Proposition 4.28 we have

Hence $g_{\mathit{v}}^{\prime }(0)=0$ if and only if $\mathrm{grad}f(\mathit{a})=\mathbf{0}\text{.}$ Now let us look at the second order derivative. By Proposition 4.43 we know that

where $H_f(\mathit{a})$ is the Hessian matrix of $f$ at $\mathit{a}$ defined by (4.5). Hence, if

for all unit vectors $\mathit{v}\text{,}$ then $f$ attains a maximum at $\mathit{a}\text{.}$ The last conclusion is not obvious, an additional argument is required. Before we formulate a theorem we introduce some definitions from linear algebra.

To decide whether a matrix is positive definite we need to use some linear algebra. Recall that every symmetric $N\times N$ matrix $A$ is diagonalisable by means of an orthogonal matrix. An orthogonal matrix is an invertible matrix $P$ such that $P^{-1}=P^T\text{,}$ that is, $P^{T}P=P{P}^T=I\text{.}$ You learn in linear algebra that for every symmetric matrix $A$ there exists an orthogonal matrix $P$ such that

is diagonal and ${\lambda }_1,\dots ,{\lambda }_N$ are the eigenvalues of $A\text{.}$ By definition of the Euclidean norm we have

Similarly we get $\Vert P^T\mathit{x}{\Vert }^2=\Vert {\mathit{x}}^2\Vert \text{.}$ If $\mathit{y}=(y_1,\dots ,y_N)\text{,}$ then a simple calculation shows that

and hence

where

Combining all the above the above we get

for all $\mathit{x}\in {\mathbb{R}}^N\text{.}$ Note that there is equality if $\mathit{x}$ happens to be an eigenvector corresponding to ${\lambda }_{max}$ and ${\lambda }_{min}\text{,}$ respectively. This leads to the following criterion for positive and negative matrices.