5.5 Higher partial derivatives and the equality of mixed partials

Differentiate a field twice and there are n² second partials, but for any field you will meet they come in equal pairs, f_xy = f_yx. This chapter proves that (Schwarz, Clairaut) with the mean value theorem on a rectangle, shows the function where it fails, and records what the pairing buys. It also sharpens the test of 5.3 for differentiability.

Differentiate $f(x, y)$ in $x$ and the result is again a function of $(x, y)$ , so you may differentiate it again, in $x$ or in $y$ . With two variables there are four second derivatives $f_{xx}$ , $f_{xy}$ , $f_{yx}$ , $f_{yy}$ , and with $n$ variables $n^2$ . Every example so far had $f_{xy} = f_{yx}$ . This chapter asks whether that is a law, finds that it is nearly one, and proves exactly how nearly. It also uses the same mean value argument as 5.3 to sharpen the test for differentiability.

Before you start

1.4: the mean value theorem. 5.2: partial derivatives and the mixed partial computations of homework 11. 5.3: continuous partial derivatives imply differentiable, and the chain rule of 5.4, which makes second derivatives of composites work.

Quick checkThe mean value theorem says that for

g

continuous on

[a, b]

and differentiable on

(a, b)

Quick checkWhich condition on the partial derivatives of

f

near

a

guarantees that

f

is differentiable at

a

Quick checkIn this book

f_{xy}

means

(f_x)_y

. For

f(x, y) = x^2y^3

, what is

f_{xy}

The picture: one rectangle, two ways to cut it

Take a rectangle with corners at the origin, $(h, 0)$ , $(0, k)$ and $(h, k)$ , and form the signed sum of $f$ at the four corners, $\Delta(h, k) = f(h, k) - f(h, 0) - f(0, k) + f(0, 0)$ . Divide by the area $hk$ . The colour in the picture is that number. A mixed second derivative is exactly a limit of it: shrink $k$ first and you get $\frac{f_y(h, 0) - f_y(0, 0)}{h}$ , then shrink $h$ and you reach $f_{yx}(0, 0)$ ; shrink them in the other order and you reach $f_{xy}(0, 0)$ . For the smooth function the colours merge into one value as you zoom in, so the two orders give the same limit. For the other one, the colour depends only on the direction, and the two orders give $+1$ and $-1$ . Everything in the chapter is a proof of what the picture shows.

The idea, in words

The corner sum $\Delta$ can be read in two ways. Subtract along vertical edges first: it is the change in the quantity $f(x, b + k) - f(x, b)$ as $x$ moves from $a$ to $a + h$ . By the mean value theorem that change is $h$ times the $x$ -derivative at some point. The $x$ -derivative of $f(x, b + k) - f(x, b)$ is $f_x(x, b + k) - f_x(x, b)$ , again a difference over a vertical edge, and the mean value theorem in $y$ turns it into $k\,f_{xy}$ at a point. So $\Delta = hk\,f_{xy}$ at some point inside the rectangle. Subtract along horizontal edges first and the same steps give $\Delta = hk\,f_{yx}$ at some other point inside. Two points of the same small rectangle, both values equal to $\Delta/(hk)$ . If the second partials are continuous, the values at the two points get close to each other as the rectangle shrinks, and both tend to the value at the corner.

Definitions

Definition T

Higher-order partial derivative

Let $f$ be a scalar field on an open set. Its partial derivatives $D_1f, \dots, D_nf$ are again scalar fields, and may have partial derivatives. The second-order partials are $D_jD_if$ for $i, j = 1, \dots, n$ (differentiate in $x_i$ first, then in $x_j$ ), and in general a partial derivative of order $k$ is $D_{j_k}\cdots D_{j_2}D_{j_1}f$ . In the plane, with $x_1 = x$ , $x_2 = y$ : this book’s $f_{xy}$ is $D_2D_1f$ (” $x$ first, then $y$ ”), Apostol’s $D_{2,1}f = D_2(D_1f)$ and Leibniz’s $\dfrac{\partial^2f}{\partial y\,\partial x}$ . Notation conventions differ between books (and Spagnolie’s boards are not consistent), so read the definition each time. We will always say which order a statement uses; for a $C^2$ field it never matters.

In one lineA partial derivative of a partial derivative: differentiate $f$ with respect to one variable, then the result with respect to a variable, and so on $k$ times; the result is a partial derivative of order $k$ .

Why it existsSecond derivatives measure how a field bends, and higher ones are what Taylor's formula needs; they are partial derivatives of the fields that the first partials already are.

Definition T

Mixed partial derivative

A partial derivative of order $k \ge 2$ is mixed if the variables it differentiates in are not all the same. In two variables, $f_{xy}$ and $f_{yx}$ are the two mixed second-order partials.

In one lineA higher-order partial derivative in which at least two different variables occur, such as $f_{xy}$ and $f_{yx}$ in two variables.

Why it existsPure second partials like $f_{xx}$ are one-variable derivatives, while mixed ones record how the slope in one direction changes as you move in another, and that is where the order question comes from.

Definition A

Class C^k

Let $S \subseteq \mathbb R^n$ be open. A scalar field $f: S \to \mathbb R$ is of class $C^k$ if every partial derivative of order $0, 1, \dots, k$ exists and is continuous on $S$ . Class $C^0$ is “continuous”; class $C^1$ is the “continuously differentiable” of 5.3; $C^\infty$ means all orders. Polynomials, $\exp$ , $\sin$ , and sums, products, quotients and compositions of them are $C^\infty$ where defined (the partials are again such expressions, by the chain rule of 5.4). A vector field is $C^k$ if each component is. The classes get smaller as $k$ grows: $x|x|$ on $\mathbb R$ is $C^1$ (its derivative is $2|x|$ ) and not $C^2$ .

In one lineA field is of class $C^k$ on an open set if all its partial derivatives of order at most $k$ exist and are continuous there; $C^\infty$ means $C^k$ for every $k$ .

Why it existsIt is the one hypothesis that makes every theorem in this chapter and the next one true, and it extends 'continuously differentiable' ( $C^1$ ) of 5.3 to higher orders.

By the theorem of 5.3, a $C^1$ field is differentiable. If $f$ is $C^2$ then each $D_if$ is $C^1$ , hence differentiable, so the gradient field $\nabla f$ has a Jacobian matrix, whose $(i, j)$ entry is $D_jD_if$ . This matrix of second partials is what 5.6 calls the Hessian. Its symmetry, proved below, is why the spectral theorem (3.1) applies to it.

Partial derivatives and differentiability, sharpened

5.3 proved: if all the partials exist near $a$ and are continuous at $a$ , then $f$ is differentiable at $a$ . The proof used the mean value theorem on one coordinate at a time. If you look at where each hypothesis was used, one of the coordinates did not need it.

Theorem B the argument of Apostol 8.13, one step sharper

Differentiability from n minus one continuous partials

Suppose the partials $D_1f, \dots, D_nf$ exist on a ball $B(a)$ and $D_2f, \dots, D_nf$ are continuous at $a$ . Then $f$ is differentiable at $a$ and $f(a + v) = f(a) + \nabla f(a)\cdot v + \lVert v \rVert\,E(a, v)$ with $E(a, v) \to 0$ as $v \to O$ . (Which coordinate is the exceptional one does not matter; relabel.)

In one lineIf $D_1f, \dots, D_nf$ exist in a ball around $a$ , and $D_2f, \dots, D_nf$ are continuous at $a$ (while $D_1f$ merely exists at $a$ ), then $f$ is differentiable at $a$ with total derivative $v \mapsto \nabla f(a)\cdot v$ .

Why it is trueIt shows how little is needed: one partial derivative of any single coordinate need not even be continuous, so a field can be differentiable at a point where it is not $C^1$ in the sense of 5.3.

ProofApostol 8.13, adapted

The idea: Walk from a to a + v one coordinate at a time, as in 5.3. The first leg changes only the first coordinate: there the definition of the partial derivative at a gives the estimate. Every other leg uses the mean value theorem and continuity of that coordinate's partial at a.

f(a + v) - f(a) = \sum_{k=1}^n\big[f(p_k) - f(p_{k-1})\big]

Same path as in 5.3:

p_0 = a

p_k = a + (v_1, \dots, v_k, 0, \dots, 0)

, so

p_n = a + v

. Telescoping sum.

First leg:

f(p_1) - f(a) = v_1D_1f(a) + r_1

with

|r_1| \le \varepsilon|v_1|

The first leg is the function

t \mapsto f(a + te_1)

t = v_1

; its derivative at

t = 0

D_1f(a)

, which exists. Definition of the derivative, with an

\varepsilon

: there is

\delta_1 > 0

with

|f(a + v_1e_1) - f(a) - v_1D_1f(a)| \le \varepsilon|v_1|

for

|v_1| < \delta_1

Other legs:

f(p_k) - f(p_{k-1}) = v_kD_kf(c_k)

with

c_k

on the segment from

p_{k-1}

p_k

For

k \ge 2

the legs

p_{k-1} \to p_k

differ in coordinate

k

only, and

t \mapsto f(p_{k-1} + te_k)

is differentiable (the partial exists in the ball). The mean value theorem of 1.4.

For

\lVert v \rVert < \delta

|D_kf(c_k) - D_kf(a)| < \varepsilon

D_kf

is continuous at

a

for

k \ge 2

, and

\lVert c_k - a \rVert \le \lVert v \rVert

. Choose

\delta \le \delta_1

so small that

|D_kf(x) - D_kf(a)| < \varepsilon

for

\lVert x - a \rVert < \delta

and

k = 2, \dots, n

\big|f(a + v) - f(a) - \nabla f(a)\cdot v\big| \le \varepsilon|v_1| + \sum_{k \ge 2}|v_k|\,\varepsilon \le n\varepsilon\lVert v \rVert

. Since

\varepsilon

was arbitrary,

f

is differentiable at

a

\blacksquare

Add the legs and subtract

\nabla f(a)\cdot v = \sum v_kD_kf(a)

. Use

|v_k| \le \lVert v \rVert

.Each leg contributes at most ε times |v_k|; that is a factor ε times a length, as the definition of differentiable demands.

Proof workshopGive the reason for each line of the sharper differentiability proofApostol 8.13, adapted

Each line of the proof is shown in order. Choose the reason that justifies it, then press Check.

$f(a + v) - f(a) = \sum_{k=1}^n [f(p_k) - f(p_{k-1})]$ , where $p_k$ changes the first $k$ coordinates.
$f(p_1) - f(a) = v_1 D_1 f(a) + r_1$ with $|r_1| \le \varepsilon |v_1|$ .
For $k \ge 2$ , $f(p_k) - f(p_{k-1}) = v_k D_k f(c_k)$ .
$|D_k f(c_k) - D_k f(a)| < \varepsilon$ for $\lVert v \rVert$ small.
$|f(a + v) - f(a) - \nabla f(a) \cdot v| \le n \varepsilon \lVert v \rVert$ .

Worked example new (the sharper theorem at work; compare 5.3, the field $(x^2 + y^2)\sin\frac1{x^2 + y^2}$)

Show that

f(x, y) = (1 + y^2)\,x^2\sin(1/x)

for

x \ne 0

f(0, y) = 0

, is differentiable at the origin although it is not continuously differentiable there.

Solution.

The partials. For $x \ne 0$ : $f_x = (1 + y^2)\big(2x\sin\frac1x - \cos\frac1x\big)$ , $f_y = 2y\,x^2\sin\frac1x$ . On the line $x = 0$ : $f_x(0, y) = \lim_{h \to 0}\frac{(1 + y^2)h^2\sin(1/h)}{h} = 0$ , and $f_y(0, y) = 0$ . Both partials exist at every point of the plane.
$f_x$ is not continuous at the origin. Along $y = 0$ and $x = \frac1{2\pi n}$ , $f_x = -\cos(2\pi n) = -1$ , but $f_x(0, 0) = 0$ . So $f$ is not $C^1$ and the theorem of 5.3 does not apply.
$f_y$ is continuous at the origin. $|f_y(x, y)| \le 2|y|x^2 \to 0 = f_y(0, 0)$ . The sharper theorem needs $D_2f = f_y$ continuous at the point, and $D_1f = f_x$ only to exist.
Apply the theorem. $f_x$ exists in a ball around $(0, 0)$ , $f_y$ exists near it and is continuous at it, so $f$ is differentiable at $(0, 0)$ with total derivative $v \mapsto 0$ . Direct check: $|f(v)| \le (1 + y^2)x^2 \le 2\lVert v \rVert^2$ for $\lVert v \rVert \le 1$ , so $f(v) = \lVert v \rVert E$ with $E \to 0$ .

Answer. Differentiable at the origin, with $\nabla f = O$ , while $D_1f$ is discontinuous there. The 5.3 test is sufficient and not necessary, and this example shows it can be sharpened but not reversed.

Worked example Apostol 8.24 #1

Apostol 8.24 #1. Find a scalar field

f

with

D_1f(0, 0) = D_2f(0, 0) = 0

whose derivative at the origin in the direction of

\mathbf i + \mathbf j

exists and equals

3

. Explain why

f

cannot be differentiable at the origin.

Solution.

Try a homogeneous field of degree 1 vanishing on the axes. $f(x, y) = \dfrac{3xy(x + y)}{x^2 + y^2}$ , $f(0, 0) = 0$ . On the axes $xy = 0$ , so $f = 0$ there and both partials at the origin are $0$ .
Direction $(1, 1)$ . $f(h, h) = \dfrac{3h\cdot h\cdot2h}{2h^2} = 3h$ , so $\dfrac{f(h, h) - f(0)}{h} = 3$ for every $h \ne 0$ , and $f'(0; (1, 1)) = 3$ . The limit is the constant $3$ .
Why not differentiable. If $f$ were differentiable at $O$ , then by 5.3 $f'(O; y) = \nabla f(O)\cdot y$ for every $y$ , which for $y = (1, 1)$ is $0 + 0 = 0 \ne 3$ . The formula of 5.3 requires differentiability, and here it gives the wrong value.

Answer. $f = \dfrac{3xy(x + y)}{x^2 + y^2}$ . In a general direction $(a, b)$ the derivative is $\dfrac{3ab(a + b)}{a^2 + b^2}$ , which exists but is not linear in $(a, b)$ : the same warning as in 5.2.

Worked example Apostol 8.24 #3

Apostol 8.24 #3. Let

f(x, y) = \dfrac{xy^3}{x^2 + y^6}

for

(x, y) \ne (0, 0)

and

f(0, 0) = 0

. Show that

f'(0; a)

exists for every vector

a

and find it, and decide whether

f

is continuous at the origin.

Solution.

Directions. For $a = (a_1, a_2)$ : $f(ha) = \dfrac{h^4a_1a_2^3}{h^2a_1^2 + h^6a_2^6}$ , so $\dfrac{f(ha) - f(0)}{h} = \dfrac{h\,a_1a_2^3}{a_1^2 + h^4a_2^6}$ . If $a_1 \ne 0$ this tends to $0$ as $h \to 0$ . If $a_1 = 0$ , then $f(ha) = 0$ for all $h$ . Divide numerator and denominator by $h^2$ .
Conclusion for directions. $f'(0; a) = 0$ for every $a$ . Unlike the example of 5.2, this directional derivative is a linear function of $a$ (the zero function): the gradient is $O$ and $f'(0; a) = \nabla f(0)\cdot a$ for all $a$ .
Continuity. Along the curve $x = y^3$ : $f(y^3, y) = \dfrac{y^3\cdot y^3}{y^6 + y^6} = \tfrac12$ , but $f(0, 0) = 0$ . Points with $f = \frac12$ come arbitrarily close to the origin.

Answer. $f'(0; a) = 0$ for all $a$ , yet $f$ is not continuous at the origin, hence not differentiable. So even “all directional derivatives exist and equal $\nabla f\cdot a$ ” does not give differentiability: the approximation must be good along curves too, not only along lines.

How many partial derivatives are there?

In two variables a function has $2^k$ ordered ways to differentiate $k$ times. For a function of $n$ variables there are $n^k$ ordered strings. We will prove that for $C^k$ fields only the number of times each variable occurs matters. That is a choice of a multiset of size $k$ from $n$ variables, and the number of those is $\binom{n + k - 1}{k}$ (put $k$ identical balls into $n$ boxes: $k$ balls and $n - 1$ dividers in a row). For $n = 3$ , $k = 3$ it is $\binom53 = 10$ instead of $27$ ; for $n = 2$ , $k = 4$ it is $5$ instead of $16$ .

Your answerHow many different third-order partial derivatives does a

C^3

function of three variables have, once the order of differentiation does not matter?

A function whose mixed partials disagree

Before the theorem, the example it has to survive.

Worked example Apostol 8.23, 8.24 #2; Lecture 24 (the remark on continuity)

Let

f(x, y) = \dfrac{xy(x^2 - y^2)}{x^2 + y^2}

for

(x, y) \ne (0, 0)

and

f(0, 0) = 0

. Compute

f_x(0, y)

and

f_y(x, 0)

, and deduce

f_{xy}(0, 0)

and

f_{yx}(0, 0)

Solution.

$f_x$ on the $y$ -axis, from the definition. For $y \ne 0$ : $\dfrac{f(h, y) - f(0, y)}{h} = \dfrac{h\,y\,(h^2 - y^2)}{(h^2 + y^2)\,h} = \dfrac{y(h^2 - y^2)}{h^2 + y^2} \to -y$ as $h \to 0$ . At $y = 0$ , $f(h, 0) = 0$ so $f_x(0, 0) = 0 = -0$ . $f(0, y) = 0$ for all $y$ , so the numerator is $f(h, y)$ alone; cancel the $h$ . So $f_x(0, y) = -y$ for every $y$ .
$f_y$ on the $x$ -axis. For $x \ne 0$ : $\dfrac{f(x, k) - f(x, 0)}{k} = \dfrac{x(x^2 - k^2)}{x^2 + k^2} \to x$ as $k \to 0$ , and $f_y(0, 0) = 0$ . Same computation with the roles swapped. So $f_y(x, 0) = x$ for every $x$ .
Differentiate once more. $f_{xy}(0, 0) = \dfrac{d}{dy}f_x(0, y)\Big|_{y = 0} = \dfrac{d}{dy}(-y) = -1$ and $f_{yx}(0, 0) = \dfrac{d}{dx}f_y(x, 0)\Big|_{x = 0} = \dfrac{d}{dx}(x) = +1$ . $f_{xy}$ means $(f_x)_y$ : take $f_x$ along the $y$ -axis, where we know it, and differentiate in $y$ . Likewise for $f_{yx}$ .
Cross-check with the rectangle. Since $f = 0$ on both axes, $\Delta(h, k) = f(h, k)$ and $\Delta/(hk) = \dfrac{h^2 - k^2}{h^2 + k^2}$ . Letting $k \to 0$ first gives $1$ (then $h \to 0$ : $f_{yx}(0, 0) = 1$ ), letting $h \to 0$ first gives $-1$ . Same two limits as in the picture.

Answer. $f_{xy}(0, 0) = -1 \ne +1 = f_{yx}(0, 0)$ (Apostol: $D_{2,1}f(0, 0) = -1$ , $D_{1,2}f(0, 0) = 1$ ). The mixed partials exist at the origin and differ.

Your answerFor the function above, compute

\Delta(h, k)/(hk)

(h, k) = (3t, t)

for any

t \ne 0

. (Give a decimal.)

What goes wrong? The second partials exist everywhere, but not as continuous functions.

Worked example Apostol 8.23 (the mixed partials 'are not both continuous at the origin')

For the same

f

, find

f_{xy}(x, y)

at points

(x, y) \ne (0, 0)

and show it is not continuous at the origin; show also that

f

is of class

C^1

Solution.

Partials away from the origin. $f_x = \dfrac{y(x^4 + 4x^2y^2 - y^4)}{(x^2 + y^2)^2}$ and $f_y = \dfrac{x(x^4 - 4x^2y^2 - y^4)}{(x^2 + y^2)^2}$ . Quotient rule; verified symbolically.
They are continuous at the origin. $|x^4 + 4x^2y^2 - y^4| \le x^4 + 4x^2y^2 + y^4 = (x^2 + y^2)^2 + 2x^2y^2 \le \tfrac32(x^2 + y^2)^2$ , so $|f_x| \le \tfrac32|y| \to 0 = f_x(0, 0)$ , and likewise $|f_y| \le \tfrac32|x|$ . $2x^2y^2 \le \frac12(x^2 + y^2)^2$ . So $f$ is $C^1$ on the plane.
The mixed partial away from the origin. Differentiating $f_x$ in $y$ gives $f_{xy} = \dfrac{(x^2 - y^2)(x^4 + 10x^2y^2 + y^4)}{(x^2 + y^2)^3}$ , and differentiating $f_y$ in $x$ gives the same expression, so $f_{yx} = f_{xy}$ there. Verified by computer algebra; the numerators agree.
Along the axes. On the $x$ -axis ( $y = 0$ ): $f_{xy} = \dfrac{x^2\cdot x^4}{x^6} = 1$ . On the $y$ -axis ( $x = 0$ ): $f_{xy} = \dfrac{(-y^2)\,y^4}{y^6} = -1$ . The limit of $f_{xy}$ at the origin depends on the direction: it does not exist.

Answer. $f_{xy}$ is not continuous at $(0, 0)$ (nor is $f_{yx}$ , which is the same function off the origin), and $f$ is $C^1$ but not $C^2$ there. In the next section’s theorem it is exactly this continuity that is missing.

Your answerFor the same

f

, what is the limit of

f_{xy}(0, y)

y \to 0

Quick checkBoth mixed partials of

f = \frac{xy(x^2 - y^2)}{x^2 + y^2}

exist at the origin and are different. What does this tell us?

Schwarz’s theorem

Everything in the picture comes from one identity about the rectangle.

Lemma A the argument of Apostol 8.23

The rectangle lemma

Let $S$ be open, $f_x$ and $f_{xy} = (f_x)_y$ exist on $S$ , and the closed rectangle $R$ with opposite corners $(a, b)$ and $(a + h, b + k)$ ( $h, k \ne 0$ ) lie in $S$ . Put $\Delta = f(a + h, b + k) - f(a + h, b) - f(a, b + k) + f(a, b)$ . Then there is $(\xi, \eta)$ in the interior of $R$ with $\Delta = hk\,f_{xy}(\xi, \eta)$ . By the same argument with the roles of $x$ and $y$ swapped, if $f_y$ and $f_{yx}$ exist on $S$ , then $\Delta = hk\,f_{yx}(\xi', \eta')$ for some $(\xi', \eta')$ in the interior of $R$ .

In one lineIf $f_x$ and $f_{xy}$ exist on an open set containing the rectangle with corners $(a, b)$ and $(a + h, b + k)$ , then $\Delta(h, k) = hk\,f_{xy}(\xi, \eta)$ for some point $(\xi, \eta)$ strictly inside the rectangle.

Why it is trueIt converts a combination of four values of $f$ into a second derivative at an unknown interior point, which is how a limit of rectangles becomes a mixed partial.

ProofApostol 8.23

The idea: Group the four corners in pairs along vertical edges to get a one-variable function G with Delta equal to G(a + h) minus G(a). Mean value theorem in x, then mean value theorem in y.

Put

G(x) = f(x, b + k) - f(x, b)

. Then

\Delta = G(a + h) - G(a)

and

G'(x) = f_x(x, b + k) - f_x(x, b)

Pair the corners with the same

x

-coordinate.

G

is differentiable on the closed interval between

a

and

a + h

since

f_x

exists, so it is continuous there.

\Delta = h\,G'(\xi) = h\big[f_x(\xi, b + k) - f_x(\xi, b)\big]

for some

\xi

strictly between

a

and

a + h

The mean value theorem for

G

on the interval between

a

and

a + h

(1.4).

The bracket equals

k\,f_{xy}(\xi, \eta)

for some

\eta

strictly between

b

and

b + k

The function

y \mapsto f_x(\xi, y)

is differentiable between

b

and

b + k

with derivative

f_{xy}(\xi, y)

(this is what

f_{xy}

means). Mean value theorem again.

\Delta = hk\,f_{xy}(\xi, \eta)

Combine the last two steps. For the second statement, swap

x

and

y

: use

H(y) = f(a + h, y) - f(a, y)

, for which

\Delta = H(b + k) - H(b)

\blacksquare

Theorem A Apostol Thm 8.12; Spagnolie Lecture 24

Equality of mixed partials (Schwarz, Clairaut)

Let $S \subseteq \mathbb R^2$ be open with $(a, b) \in S$ , and suppose the partial derivatives $f_x, f_y, f_{xy}, f_{yx}$ exist on $S$ . If $f_{xy}$ and $f_{yx}$ are both continuous at the point $(a, b)$ , then $f_{xy}(a, b) = f_{yx}(a, b)$ .

In one lineIf $f_x$ , $f_y$ , $f_{xy}$ , $f_{yx}$ exist on an open set containing $(a, b)$ and $f_{xy}$ and $f_{yx}$ are both continuous at $(a, b)$ , then $f_{xy}(a, b) = f_{yx}(a, b)$ .

Why it is trueIt says that when the second derivatives behave (continuously), the order of differentiation does not matter, which makes the matrix of second derivatives symmetric and lets the spectral theorem decide the shape of a critical point.

ProofApostol 8.12; this is the picture above

The idea: Compute the same rectangle sum two ways with the rectangle lemma. Equate. Shrink the rectangle to the point and use continuity of both mixed partials.

Fix small

h, k \ne 0

and let

\Delta = f(a + h, b + k) - f(a + h, b) - f(a, b + k) + f(a, b)

Choose a ball around

(a, b)

inside

S

, and take

|h|, |k|

small enough that the closed rectangle with corners

(a, b)

(a + h, b + k)

lies in the ball. Then the rectangle lemma applies.

\Delta = hk\,f_{xy}(\xi, \eta)

for some

(\xi, \eta)

in the open rectangle.

Rectangle lemma, first form (needs

f_x

f_{xy}

\Delta = hk\,f_{yx}(\xi', \eta')

for some other point

(\xi', \eta')

in the same open rectangle.

Rectangle lemma, second form (needs

f_y

f_{yx}

f_{xy}(\xi, \eta) = f_{yx}(\xi', \eta')

Divide by

hk \ne 0

Let

(h, k) \to (0, 0)

f_{xy}(a, b) = f_{yx}(a, b)

\blacksquare

Both points lie within

|h|

and

|k|

(a, b)

, so they tend to

(a, b)

(h, k) \to (0, 0)

; the left side tends to

f_{xy}(a, b)

and the right side to

f_{yx}(a, b)

, by continuity at

(a, b)

of each. A limit is unique.Only continuity at the single point (a, b) is used, not in a neighbourhood, and the proof needs both mixed partials.

Proof workshopRebuild the proof of Schwarz's theoremApostol 8.12

Tap the lines in the order a proof would write them. Each line can only use what is already above it.

Theorem A Apostol Thm 8.13; Peano (1890)

Let $f_x$ , $f_y$ and $f_{xy}$ exist on an open set containing $(a, b)$ , with $f_{xy}$ continuous at $(a, b)$ . Then $f_{yx}(a, b)$ exists and $f_{yx}(a, b) = f_{xy}(a, b)$ .

In one lineIf $f_x$ , $f_y$ and $f_{xy}$ exist on an open set containing $(a, b)$ and $f_{xy}$ is continuous at $(a, b)$ , then $f_{yx}(a, b)$ exists and equals $f_{xy}(a, b)$ .

Why it is trueIt says only one of the two mixed partials needs to be continuous: the other is then forced to exist and to agree.

ProofApostol 8.13 (the proof there is longer)

The idea: The rectangle lemma gives the quotient Δ/(hk) as a value of the continuous function f_xy near the corner, so it is close to f_xy(a, b). Letting k go to zero turns the quotient into a difference quotient of f_y, which is the definition of f_yx.

Choose

\delta

so that

f_{xy}

is within

\varepsilon

f_{xy}(a, b)

on the

\delta

-ball.

Let

\varepsilon > 0

. Continuity of

f_{xy}

(a, b)

gives

\delta > 0

with

|f_{xy}(p) - f_{xy}(a, b)| < \varepsilon

whenever

\lVert p - (a, b) \rVert < \delta

For

0 < |h|, |k| < \delta/2

\Big|\dfrac{\Delta(h, k)}{hk} - f_{xy}(a, b)\Big| < \varepsilon

Rectangle lemma:

\Delta(h, k)/(hk) = f_{xy}(\xi, \eta)

with

(\xi, \eta)

inside the rectangle, which lies in the

\delta

-ball when

|h|, |k| < \delta/2

Fix

h

and let

k \to 0

\Big|\dfrac{f_y(a + h, b) - f_y(a, b)}{h} - f_{xy}(a, b)\Big| \le \varepsilon

for

0 < |h| < \delta/2

f_y

exists at

(a + h, b)

and at

(a, b)

, so

\lim_{k \to 0}\Delta(h, k)/k = f_y(a + h, b) - f_y(a, b)

(definition of

f_y

applied to each of the two columns of

\Delta

f_{yx}(a, b) = \lim_{h \to 0}\dfrac{f_y(a + h, b) - f_y(a, b)}{h}

exists and equals

f_{xy}(a, b)

Definition of the limit: the difference quotient of

f_y

(a, b)

in the

x

-direction is within

\varepsilon

f_{xy}(a, b)

for all small

h

, for every

\varepsilon

\blacksquare

Corollary A Apostol 8.23 and Lecture 24

Order does not matter for C^k fields

Let $f$ be $C^k$ on an open set $S$ . If two strings of $k$ differentiations $(i_1, \dots, i_k)$ and $(j_1, \dots, j_k)$ use each variable the same number of times, then $D_{i_1}\cdots D_{i_k}f = D_{j_1}\cdots D_{j_k}f$ on $S$ . In particular for $k = 2$ the matrix $[D_jD_if]$ of a $C^2$ field is symmetric.

In one lineIf $f$ is of class $C^k$ on an open set, then any two partial derivatives of order at most $k$ that differentiate the same number of times in each variable are equal; in particular the matrix of second partials of a $C^2$ field is symmetric.

Why it is trueIt lets you write $\dfrac{\partial^3f}{\partial x^2\partial y}$ without saying in what order, and it is the reason the third-order Taylor term has the coefficient 3 in $3F_{xxy}x^2y$ .

Proofstandard

The idea: Any rearrangement of a string is a sequence of swaps of two neighbours. A swap of two neighbouring differentiations is Schwarz's theorem applied to the partial derivative that has already been taken, which still has two continuous derivatives.

For two differentiations,

D_jD_if = D_iD_jf

by Schwarz’s theorem.

k = 2

: every second partial of a

C^2

field is continuous, so the hypotheses of Schwarz's theorem hold at every point (the first and second partials exist by

C^2

For general

k

, it suffices to show that swapping two adjacent differentiations does not change the result.

Rearranging a list into another with the same entries is a product of swaps of adjacent entries.

Let

\varphi

be the partial derivative applied before the two neighbours; it is of class

C^2

Write the string as

D_{i_1}\cdots D_{i_{s-1}}\,(D_{i_s}D_{i_{s+1}})\,\varphi

where

\varphi = D_{i_{s+2}}\cdots D_{i_k}f

is the part applied first. This

\varphi

is a partial derivative of order

k - s - 1 \le k - 2

of a

C^k

field, so it is

C^2

So the two orders agree.

Schwarz's theorem for

\varphi

D_{i_s}D_{i_{s+1}}\varphi = D_{i_{s+1}}D_{i_s}\varphi

. The remaining outer differentiations are then applied to equal fields.

\blacksquare

Quick checkWhich hypothesis at

(a, b)

guarantees

f_{xy}(a, b) = f_{yx}(a, b)

Quick checkHow many different fourth-order partial derivatives does a

C^4

function of two variables have, once order does not matter?

Worked example Apostol 8.9 #10 to #12 style

Verify

f_{xy} = f_{yx}

for

f(x, y) = \cos(x^2)/y

(with

y \ne 0

), and find

f_{xyy}

Solution.

$x$ first. $f_x = -\dfrac{2x\sin(x^2)}{y}$ , then $f_{xy} = \dfrac{2x\sin(x^2)}{y^2}$ . Chain rule on $\cos(x^2)$ ; then differentiate $y^{-1}$ .
$y$ first. $f_y = -\dfrac{\cos(x^2)}{y^2}$ , then $f_{yx} = \dfrac{2x\sin(x^2)}{y^2}$ . The $x$ -derivative of $\cos(x^2)$ is $-2x\sin(x^2)$ , and there is an extra minus sign from $-\cos$ .
Third order. $f_{xyy} = \partial_y\Big(\dfrac{2x\sin(x^2)}{y^2}\Big) = -\dfrac{4x\sin(x^2)}{y^3}$ . The other orders $f_{yxy}$ and $f_{yyx}$ give the same by the corollary; for instance $f_{yy} = \frac{2\cos(x^2)}{y^3}$ and $\partial_x$ of it is $-\frac{4x\sin(x^2)}{y^3}$ .

Answer. $f_{xy} = f_{yx} = \dfrac{2x\sin(x^2)}{y^2}$ and $f_{xyy} = f_{yyx} = -\dfrac{4x\sin(x^2)}{y^3}$ . The function is $C^\infty$ for $y \ne 0$ , so the theorem applies at every point.

Your answerFor

f(x, y) = x^2y^3

, what is

f_{xy}(1, 2)

Gradient fields and the necessary condition

Suppose a vector field $g = (g_1, g_2)$ is the gradient of some $C^2$ function $f$ : $g_1 = f_x$ , $g_2 = f_y$ . Then $D_2g_1 = f_{xy}$ and $D_1g_2 = f_{yx}$ , which are equal by Schwarz’s theorem. So $D_2g_1 = D_1g_2$ is a necessary condition for $g$ to be a gradient. (Clairaut used exactly this equality in 1740, to find a condition that $P\,dx + Q\,dy$ must satisfy to be the differential of a function.)

Worked example Practice Final 2013, problem 11; Apostol 8.12 and 9.1

Practice Final 2013, extra credit. Suppose

g: \mathbb R^2 \to \mathbb R^2

C^1

. Is

g

necessarily the gradient of some scalar field? Test

g = (2xy + 3,\ x^2 - 1)

and

g = (-y, x)

Solution.

Necessary condition. If $g = \nabla f$ with $f$ of class $C^2$ , then $D_2g_1 = D_1g_2$ . Both are $f_{xy} = f_{yx}$ .
Test $g = (-y, x)$ . $D_2g_1 = -1$ and $D_1g_2 = +1$ . They differ, so $g$ is not a gradient. It is the swirl field of 5.1; going once around the origin with it never returns to the same height.
Test $g = (2xy + 3, x^2 - 1)$ . $D_2g_1 = 2x = D_1g_2$ . The condition holds. It is necessary, not yet sufficient.
Find the potential. Integrate $f_x = 2xy + 3$ in $x$ : $f = x^2y + 3x + c(y)$ . Then $f_y = x^2 + c'(y)$ must equal $x^2 - 1$ , so $c' = -1$ , $c = -y$ (plus a constant). So $f = x^2y + 3x - y$ . The condition made the $x$ -dependence of $c'$ cancel, which is exactly what the integration needed.

Answer. No: a $C^1$ field need not be a gradient, since $D_2g_1 = D_1g_2$ can fail. For $g = (2xy + 3, x^2 - 1)$ the answer is yes, $f = x^2y + 3x - y$ .

Your answerThe field

g = (2xy + 3,\ x^2 - 1)

is the gradient of a function

f

with

f(0, 0) = 0

. What is

f(1, 2)

Beyond the course: for understanding· The condition is sufficient on a rectangle, and not on a punctured plane

For a $C^1$ field on a rectangle or a disc centred at the origin (a region where the path from $(0, 0)$ along the $x$ -axis and then straight up stays inside), $D_2g_1 = D_1g_2$ is also sufficient: $f(x, y) = \int_0^xg_1(t, 0)\,dt + \int_0^yg_2(x, s)\,ds$ has $f_y = g_2$ and, differentiating under the integral and using the condition, $f_x = g_1(x, 0) + \int_0^yD_2g_1(x, s)\,ds = g_1(x, y)$ . On the punctured plane it fails. The field $g = \dfrac{(-y, x)}{x^2 + y^2}$ satisfies $D_2g_1 = D_1g_2 = \dfrac{y^2 - x^2}{(x^2 + y^2)^2}$ everywhere defined, but it is the gradient of the angle $\theta$ , which cannot be defined as a single-valued function on the whole punctured plane (the angle grows by $2\pi$ as you walk once around the origin). Whether the necessary condition is sufficient depends on the shape of the domain: that is the beginning of de Rham cohomology (MATH 551 and beyond).

Where this came from· From Euler and Clairaut to Schwarz

In 1740 both Euler and Clairaut published a proposed proof that the order of differentiation does not matter. Clairaut’s use for it was the question of when an expression $P\,dx + Q\,dy$ comes from a function: he showed that $P_y = Q_x$ is then needed. Those arguments were not rigorous. Lagrange gave a proof in 1797, which Cauchy improved in 1823, though it still assumed that the pure second partials $f_{xx}$ and $f_{yy}$ exist and are continuous; further proofs in the 1840s to 1860s were incomplete, and in 1867 Lindelöf gave an explicit example in which the two mixed partials are unequal. Hermann Schwarz gave the first rigorous proof in 1873. Giuseppe Peano’s short note of 1890 improved the theorem to the form of this chapter (Apostol’s 8.13: one mixed partial continuous forces the other to exist and agree). The function $\frac{xy(x^2 - y^2)}{x^2 + y^2}$ is the standard example, usually called Peano’s. The problem all of them were solving was how to recognise a gradient, and the answer, equality of mixed partials, turned out to need a hypothesis that nobody had thought to state.

Where it is used

Used inside maths· Inside maths: a symmetric Hessian, and the shape of the next chapter

For a $C^2$ field the matrix $H = [D_jD_if]$ of second partials is symmetric, so the spectral theorem (3.1) gives it real eigenvalues and orthogonal eigenvectors, and its quadratic form $v^{\mathsf T}Hv$ can be put in the form $\sum\lambda_iw_i^2$ by a rotation (3.2). That is the entire plan of 5.6: the second-order Taylor formula uses $v^{\mathsf T}Hv$ , and the sign of that form, read off from the eigenvalues, decides whether a critical point is a maximum, a minimum or a saddle. The Taylor coefficients of order $k$ in $n$ variables are the $\binom{n + k - 1}{k}$ distinct partials of order $k$ , each counted with the number of orderings that give it (the factor $3$ in $3F_{xxy}x^2y$ of Lecture 24 is $3$ orderings of $x, x, y$ ). And equality of mixed partials is what let 5.4 collect $f_{xy} + f_{yx}$ into $2f_{xy}$ in the wave-equation and Laplace computations.

Used in the real world· Physics and AI: Maxwell's relations, and the cost of a Hessian

In thermodynamics the Helmholtz free energy $F(T, V)$ of a gas has $dF = -S\,dT - p\,dV$ , which says $S = -F_T$ and $p = -F_V$ . Equality of the mixed partials of $F$ gives the Maxwell relation $\big(\frac{\partial S}{\partial V}\big)_T = \big(\frac{\partial p}{\partial T}\big)_V$ . Entropy cannot be measured with an instrument, but pressure against temperature at fixed volume can. For one mole of an ideal gas, $p = RT/V$ , so $\big(\frac{\partial p}{\partial T}\big)_V = R/V$ ; the relation then gives $\big(\frac{\partial S}{\partial V}\big)_T = R/V$ , and doubling the volume at constant temperature raises the entropy by $R\ln2 = 8.314\times0.693 \approx 5.76\ \text{J/K}$ . At $V = 0.025\ \text{m}^3$ the slope is $R/V = 332.6\ \text{Pa/K}$ . In machine learning, the loss of a model with $n = 10^6$ weights has $n^2 = 10^{12}$ second partials, but only $n(n + 1)/2 \approx 5\times10^{11}$ distinct ones because of this chapter’s theorem, which at 8 bytes each is still about 4 terabytes. That is why second-order methods never form the Hessian. They compute only the product $Hv$ of the Hessian with a vector. It is the derivative in $t$ of $\nabla f(x + tv)$ , which two gradient evaluations approximate and automatic differentiation computes exactly for a small multiple of the cost of one gradient. Reading it as the gradient of the scalar $\nabla f\cdot v$ uses the symmetry of $H$ : that gradient is $H^{\mathsf T}v$ , which equals $Hv$ by this chapter’s theorem.

Python labTwo limits that do not commute

import numpy as np

def peano(x, y):
    d = x**2 + y**2
    return 0.0 if d == 0 else x * y * (x**2 - y**2) / d

def smooth(x, y):
    return x**2 * y + np.sin(x * y)

def Q(f, h, k):                       # the rectangle quotient Delta(h, k) / (h k)
    return (f(h, k) - f(h, 0) - f(0, k) + f(0, 0)) / (h * k)

for name, f in [("smooth", smooth), ("Peano ", peano)]:
    print(name)
    print("  k -> 0 first (k = h/10^4), then h:", [round(Q(f, h, h * 1e-4), 4) for h in (1e-1, 1e-2, 1e-3)])
    print("  h -> 0 first (h = k/10^4), then k:", [round(Q(f, k * 1e-4, k), 4) for k in (1e-1, 1e-2, 1e-3)])
    print("  along the diagonal h = k:         ", [round(Q(f, t, t), 4) for t in (1e-1, 1e-2, 1e-3)])
    print("  along the direction (3, 1):       ", [round(Q(f, 3 * t, t), 4) for t in (1e-1, 1e-2, 1e-3)])

# the mixed partials at the origin: finite differences of finite differences
h = 1e-5
fx = lambda f, x, y: (f(x + h, y) - f(x - h, y)) / (2 * h)
fy = lambda f, x, y: (f(x, y + h) - f(x, y - h)) / (2 * h)
for name, f in [("smooth", smooth), ("Peano ", peano)]:
    s = 1e-3
    fxy = (fx(f, 0, s) - fx(f, 0, -s)) / (2 * s)      # x first, then y
    fyx = (fy(f, s, 0) - fy(f, -s, 0)) / (2 * s)      # y first, then x
    print(name, "f_xy(0,0) ~", round(fxy, 4), "  f_yx(0,0) ~", round(fyx, 4))

Try this. Run it. For the smooth function all four ways of shrinking the rectangle approach 1 (the direction (3, 1) gives 1.3, 1.03, 1.003): 1.1, 1.01, 1.001 along the first row, 1.0 along the second, and so on. For Peano's function the answers are +1, -1, 0 and 0.8, one for each way, and they do not move as the rectangle shrinks. The last two lines compute the mixed partials at the origin by finite differences of finite differences: for the smooth function f_xy and f_yx both come out 1; for Peano's function about -1 for f_xy and about +1 for f_yx. Change the direction (3, 1) to (1, 0.1) and check Q = (1 - 0.01)/(1 + 0.01) = 0.9802.

Traps

Watch out· Which order does the notation mean?

$\dfrac{\partial^2f}{\partial x\,\partial y}$ means differentiate in $y$ first, then in $x$ (the operator closest to $f$ acts first). The subscript $f_{xy}$ means $x$ first, then $y$ (this book, reading left to right). Apostol’s $D_{1,2}f = D_1(D_2f)$ is $y$ first. The same symbol reads in opposite orders in different books. For a $C^2$ function the difference is invisible, which is why the confusion survives; at Peano’s origin it flips the sign.

Watch out· Both mixed partials existing does not make them equal

At the origin, Peano’s function has $f_{xy} = -1$ and $f_{yx} = +1$ , both existing. Schwarz’s theorem needs continuity of the second partials at the point (both, or one by Peano’s refinement). “Exist” is not enough.

Watch out· Equal mixed partials do not mean continuous

The converse of Schwarz’s theorem is false too: a function can have $f_{xy} = f_{yx}$ at a point without the second partials being continuous there. Equality is a consequence of a hypothesis, not a test for it.

Watch out· C^1 does not give second derivatives

A $C^1$ field has first partials that are continuous; they need not be differentiable. The function $x|x|$ is $C^1$ and has no second derivative at $0$ . All the theorems of this chapter about second partials assume $C^2$ , or at least the existence of the second partials.

Watch out· Count distinct partials with the right formula

For a $C^k$ field of $n$ variables there are $\binom{n + k - 1}{k}$ distinct partials of order $k$ , not $n^k$ and not $\binom nk$ . Three variables, third order: $10$ , not $27$ and not $1$ .

Problems

checkApostol 8.9 #10 (MATH 375 HW 11)

Compute $f_{xy}$ and $f_{yx}$ for $f(x, y) = x^4 + y^4 - 4x^2y^2$ .

Solution

$f_x = 4x^3 - 8xy^2$ , so $f_{xy} = -16xy$ . $f_y = 4y^3 - 8x^2y$ , so $f_{yx} = -16xy$ . Equal, as a polynomial is $C^\infty$ .

check

How many distinct second-order partials does a $C^2$ function of 4 variables have? How many distinct fifth-order partials does a $C^5$ function of 2 variables have?

Solution

Second order, 4 variables: $\binom{4 + 2 - 1}{2} = \binom52 = 10$ (that is $n(n + 1)/2$ with $n = 4$ ). Fifth order, 2 variables: $\binom{2 + 5 - 1}{5} = \binom65 = 6$ : $f_{x^5}, f_{x^4y}, \dots, f_{y^5}$ .

checkPractice Final 2013, extra credit (the necessary condition)

Is $g(x, y) = (y^2,\ x^2)$ the gradient of a $C^2$ function? Is $g(x, y) = (x^2,\ y^2)$ ?

Solution

For $(y^2, x^2)$ : $D_2g_1 = 2y$ and $D_1g_2 = 2x$ , which differ (for example at $(1, 0)$ : $0$ against $2$ ), so it is not a gradient. For $(x^2, y^2)$ : $D_2g_1 = 0 = D_1g_2$ , and indeed it is $\nabla\big(\frac{x^3 + y^3}{3}\big)$ .

checkApostol 8.24 #2 (the same function)

For Peano’s function, state $f_x(0, 0)$ , $f_y(0, 0)$ , $f_{xy}(0, 0)$ and $f_{yx}(0, 0)$ and say which hypothesis of Schwarz’s theorem fails.

Solution

$f_x(0, 0) = f_y(0, 0) = 0$ (since $f$ vanishes on both axes), $f_{xy}(0, 0) = -1$ and $f_{yx}(0, 0) = +1$ . The second partials exist but are not continuous at $(0, 0)$ : $f_{xy}$ tends to $1$ along the $x$ -axis and to $-1$ along the $y$ -axis. Both continuity hypotheses fail (and by Peano’s refinement, so does the hypothesis on $f_{xy}$ alone).

useApostol 8.14 #10 style, with 5.4

Let $f$ be $C^2$ and let $g(t) = f(t, t^2)$ . Find $g''(t)$ in terms of the partials of $f$ , and evaluate at $t = 1$ for $f = x^2y + xy^3$ .

Solution

By the second-derivative rule of 5.4 with $r = (t, t^2)$ , $r' = (1, 2t)$ , $r'' = (0, 2)$ : $g'' = f_{xx} + 4t\,f_{xy} + 4t^2f_{yy} + 2f_y$ (using $f_{xy} = f_{yx}$ ). For $f = x^2y + xy^3$ : $f_{xx} = 2y$ , $f_{xy} = 2x + 3y^2$ , $f_{yy} = 6xy$ , $f_y = x^2 + 3xy^2$ . At $(1, 1)$ : $g''(1) = 2 + 4\cdot5 + 4\cdot6 + 2\cdot4 = 54$ . Check: $g = t^2\cdot t^2 + t\cdot t^6 = t^4 + t^7$ , $g'' = 12t^2 + 42t^5$ , which at $t = 1$ is $54$ . ✓

use

Find a function $f$ with $f_x = 2xy^2 + \cos x$ and $f_y = 2x^2y + 1$ , or show there is none.

Solution

Test: $D_2(2xy^2 + \cos x) = 4xy$ and $D_1(2x^2y + 1) = 4xy$ . Equal, so a potential may exist. Integrate $f_x$ in $x$ : $f = x^2y^2 + \sin x + c(y)$ . Then $f_y = 2x^2y + c'(y)$ must equal $2x^2y + 1$ , so $c = y$ . Hence $f = x^2y^2 + \sin x + y$ (plus a constant).

use

Show that $f(x, y) = \dfrac{x^3y}{x^2 + y^2}$ , $f(0, 0) = 0$ , is differentiable at the origin, find its gradient there, and find $f_{xy}(0, 0)$ and $f_{yx}(0, 0)$ . Is $f$ of class $C^2$ ?

Solution

$|f| \le \dfrac{|x|\cdot x^2|y|}{x^2 + y^2} \le |x||y| \le \frac12\lVert v \rVert^2$ , so $f(v) = \lVert v \rVert E(v)$ with $E \to 0$ : differentiable at $O$ with $\nabla f(O) = O$ . Away from the origin $f_x = \dfrac{y(3x^2(x^2 + y^2) - 2x^4)}{(x^2 + y^2)^2} = \dfrac{y\,x^2(x^2 + 3y^2)}{(x^2 + y^2)^2}$ and $f_y = \dfrac{x^3(x^2 - y^2)}{(x^2 + y^2)^2}$ . On the $y$ -axis $f_x(0, y) = 0$ , so $f_{xy}(0, 0) = 0$ . On the $x$ -axis $f_y(x, 0) = x$ , so $f_{yx}(0, 0) = 1$ . They differ, so $f$ is not $C^2$ at the origin.

provenew (uses 5.4)

Prove that if $f$ is $C^2$ on $\mathbb R^2$ and $f_{xy} \equiv 0$ , then $f(x, y) = \varphi(x) + \psi(y)$ for some functions $\varphi, \psi$ .

Solution

Plan. $f_x$ has zero $y$ -derivative, so $f_x$ depends on $x$ alone; integrate in $x$ .

$\partial_y(f_x) = f_{xy} = 0$ on all of $\mathbb R^2$ , so for each fixed $x$ the function $y \mapsto f_x(x, y)$ has derivative $0$ on $\mathbb R$ and is constant (1.4): $f_x(x, y) = \alpha(x)$ , where $\alpha(x) = f_x(x, 0)$ . Let $\varphi$ be an antiderivative of $\alpha$ (it exists since $\alpha$ is continuous). Then $\partial_x\big(f - \varphi(x)\big) = 0$ , so for each fixed $y$ the function $x \mapsto f(x, y) - \varphi(x)$ is constant: $f(x, y) = \varphi(x) + \psi(y)$ with $\psi(y) = f(0, y) - \varphi(0)$ . $\blacksquare$

proveApostol 8.9 #20 and #21(b)

Let $S$ be an open convex set (the segment between any two of its points lies in $S$ ). Prove that if $f'(x; y) = 0$ for every $x \in S$ and every $y$ , then $f$ is constant on $S$ . (Hint: the mean value theorem for scalar fields from 5.2.)

Solution

Plan. Join two points by a segment and apply the mean value theorem of 5.2 on it.

Let $p, q \in S$ and $y = q - p$ . The segment $\{p + ty : 0 \le t \le 1\}$ lies in $S$ by convexity. By the mean value theorem for scalar fields (5.2, Apostol 8.4), $f(q) - f(p) = f'(z; y)$ for some $z$ on the segment. By hypothesis $f'(z; y) = 0$ , so $f(q) = f(p)$ . Since $p, q$ were arbitrary, $f$ is constant. $\blacksquare$ (Every ball is convex, because $\lVert tp + (1 - t)q - a \rVert \le t\lVert p - a \rVert + (1 - t)\lVert q - a \rVert < r$ , which recovers the ball case of 5.2.)

prove

Prove that if $f$ is $C^3$ then $f_{xyx} = f_{xxy} = f_{yxx}$ (in the book’s order: the first letter is the first differentiation).

Solution

Plan. Swap neighbours one at a time, each swap being Schwarz’s theorem for a $C^2$ field.

In operator form $f_{xyx} = D_xD_yD_xf$ , $f_{xxy} = D_yD_xD_xf$ and $f_{yxx} = D_xD_xD_yf$ . Put $\varphi = D_xf$ , which is $C^2$ . Schwarz’s theorem for $\varphi$ gives $D_xD_y\varphi = D_yD_x\varphi$ , that is, $f_{xyx} = f_{xxy}$ . Schwarz’s theorem for $f$ gives $D_xD_yf = D_yD_xf$ everywhere; apply $D_x$ to both sides: $D_xD_xD_yf = D_xD_yD_xf$ , that is, $f_{yxx} = f_{xyx}$ . $\blacksquare$ (In general, any two orders with the same counts agree: this is the corollary.)

provenew (a necessary condition)

Prove that if $g = (g_1, \dots, g_n)$ is the gradient of a $C^2$ scalar field, then $D_jg_i = D_ig_j$ for all $i, j$ ; equivalently, the Jacobian matrix of $g$ is symmetric.

Solution

Plan. Write the entries as second partials of $f$ and apply Schwarz’s theorem.

If $g = \nabla f$ , then $g_i = D_if$ , so $D_jg_i = D_jD_if$ and $D_ig_j = D_iD_jf$ . These are second partials of a $C^2$ field, and equal by Schwarz’s theorem. The $(i, j)$ entry of the Jacobian is $D_jg_i$ and the $(j, i)$ entry is $D_ig_j$ , so the Jacobian is symmetric. $\blacksquare$

stretchnew (the picture of this chapter as a theorem)

Let $f_x$ and $f_{xy}$ exist near $(a, b)$ with $f_{xy}$ continuous at $(a, b)$ . Prove that the double limit $\displaystyle\lim_{(h, k) \to (0, 0)}\frac{\Delta(h, k)}{hk}$ exists and equals $f_{xy}(a, b)$ . Deduce that for Peano’s function $f_{xy}$ cannot be continuous at the origin.

Solution

Plan. The rectangle lemma writes the quotient as a value of $f_{xy}$ at a point of the rectangle.

For small $h, k \ne 0$ the lemma gives $\Delta(h, k)/(hk) = f_{xy}(\xi, \eta)$ with $(\xi, \eta)$ within $|h|$ and $|k|$ of $(a, b)$ . Given $\varepsilon > 0$ , continuity of $f_{xy}$ at $(a, b)$ gives $\delta$ with $|f_{xy}(p) - f_{xy}(a, b)| < \varepsilon$ for $\lVert p - (a, b) \rVert < \delta$ , so $|\Delta/(hk) - f_{xy}(a, b)| < \varepsilon$ whenever $\sqrt{h^2 + k^2} < \delta/2$ . $\blacksquare$ For Peano’s function $\Delta/(hk) = \frac{h^2 - k^2}{h^2 + k^2}$ is $0$ along $h = k$ and $0.8$ along $(3t, t)$ , so the double limit does not exist; by the theorem $f_{xy}$ is therefore not continuous at the origin, which we also saw directly.

stretch

Let $f$ be $C^2$ on a disc. Show that the second derivative of $f$ in the direction of a unit vector $u = (\cos\theta, \sin\theta)$ , defined as $\dfrac{d^2}{dt^2}f(a + tu)\big|_{t = 0}$ , equals $f_{xx}\cos^2\theta + 2f_{xy}\sin\theta\cos\theta + f_{yy}\sin^2\theta$ . For which $\theta$ is it largest, and what is that largest value in terms of the matrix $\begin{bmatrix}f_{xx} & f_{xy}\\ f_{xy} & f_{yy}\end{bmatrix}$ ?

Solution

By the second-derivative rule along a path (5.4) with $r(t) = a + tu$ , $r' = u$ , $r'' = O$ : $\frac{d^2}{dt^2}f(a + tu) = u^{\mathsf T}Hu$ with $H$ the matrix of second partials, which is the displayed formula (using $f_{xy} = f_{yx}$ ). By the spectral theorem ( $H$ is symmetric), the maximum of $u^{\mathsf T}Hu$ over unit vectors is the largest eigenvalue of $H$ , attained at the corresponding eigenvector (3.2). So the direction of greatest curvature is an eigenvector of the largest eigenvalue, and the greatest curvature is that eigenvalue. This is the idea of 5.6.

Summary

A higher-order partial derivative is a partial derivative of a partial derivative; a mixed one differentiates in at least two different variables. A field is of class $C^k$ if all partials of order at most $k$ exist and are continuous. $C^1$ implies differentiable (5.3), and $C^2$ makes the gradient differentiable with the matrix of second partials as its Jacobian.
Sharpening 5.3. Existence of all partials near $a$ with $n - 1$ of them continuous at $a$ already gives differentiability at $a$ (proved with the mean value theorem on one coordinate at a time). It is still sufficient and not necessary.
Schwarz’s theorem. If $f_x, f_y, f_{xy}, f_{yx}$ exist near $(a, b)$ and $f_{xy}, f_{yx}$ are continuous at $(a, b)$ , then $f_{xy}(a, b) = f_{yx}(a, b)$ . Proof: the rectangle sum $\Delta$ equals $hk\,f_{xy}$ at one interior point and $hk\,f_{yx}$ at another (mean value theorem twice, in two orders), so the two values get close as the rectangle shrinks. Peano’s refinement: continuity of one mixed partial forces the other to exist and agree.
The counterexample. $f = \frac{xy(x^2 - y^2)}{x^2 + y^2}$ has $f_x(0, y) = -y$ and $f_y(x, 0) = x$ , hence $f_{xy}(0, 0) = -1 \ne +1 = f_{yx}(0, 0)$ . It is $C^1$ and not $C^2$ : $f_{xy}$ tends to $1$ along the $x$ -axis and to $-1$ along the $y$ -axis. The rectangle quotient $\frac{\Delta}{hk} = \frac{h^2 - k^2}{h^2 + k^2}$ depends on direction only.
Order does not matter for $C^k$ fields; the number of distinct partials of order $k$ in $n$ variables is $\binom{n + k - 1}{k}$ . The matrix of second partials of a $C^2$ field is symmetric, so the spectral theorem applies to it (5.6).
Gradient fields. $D_jg_i = D_ig_j$ is necessary for $g$ to be a gradient (Practice Final extra credit: $(-y, x)$ is not one) and sufficient on a rectangle or disc, though not on the punctured plane.

Watch and read

The same ideas explained by someone else. Use these when an explanation here does not land.

5.5 Higher partial derivatives and the equality of mixed partials

The picture: one rectangle, two ways to cut it

The idea, in words

Definitions

Higher-order partial derivative

Mixed partial derivative

Class C^k

Partial derivatives and differentiability, sharpened

Differentiability from n minus one continuous partials

How many partial derivatives are there?

A function whose mixed partials disagree

Schwarz’s theorem

The rectangle lemma

Equality of mixed partials (Schwarz, Clairaut)

Peano's refinement of Schwarz's theorem

Order does not matter for C^k fields

Gradient fields and the necessary condition

Where it is used

Traps

Problems

Summary

Watch and read

Where this sits

Comes from

Leads to

Same idea elsewhere