5.4 The chain rule for fields

If one field is fed into another, the derivative of the composite is the product of the two Jacobian matrices, the outer one evaluated at the image point. This one rule gives the derivative along a path, the formulas for changing coordinates, the derivative of an inverse (the inverse matrix), and, applied twice, the second derivatives of a composite.

A temperature depends on where you are. Where you are depends on the time. How fast does the temperature you feel change? In one variable (1.4) the answer is a product of slopes. In many variables the “slopes” are the Jacobian matrices of 5.3, and the answer is a product of matrices. This chapter proves that, and then spends the proof: the same rule gives the derivative along a path, the formulas for changing from $(x, y)$ to polar coordinates, the derivative of an inverse map, and, used twice, every second-derivative computation in the course.

Before you start

5.3: the Jacobian matrix of a differentiable field, rows are gradients. 2.5: the matrix of a composition of linear maps is the product of the matrices. 1.4: the one-variable chain rule.

Quick check

g: \mathbb R^2 \to \mathbb R^3

and

f: \mathbb R^3 \to \mathbb R^4

are differentiable. What shape is the Jacobian of

f \circ g

Quick checkTwo linear maps have matrices

A

(apply first) and

B

(apply second). What is the matrix of 'first

A

, then

B

Quick checkIn one variable,

h(x) = \sin(x^2)

. At which point is the outer derivative

\cos

evaluated?

The picture: circles into ellipses into ellipses

Drag the dot in the left plane and shrink the radius $\rho$ . A tiny circle of steps around $a$ is sent by $g$ to a curve that is almost an ellipse (the blue curve in the middle: this is the picture of 5.3, zoomed). Then $f$ sends that ellipse to another one. The right-hand picture compares the true image of the circle under the composite $f \circ g$ (purple) with the unit circle sent by the product $Df(g(a))\,Dg(a)$ (green dashed). The “largest gap” in the box under the picture falls towards $0$ with $\rho$ . The two ellipses are the same shape once the circle is small: a composite of two things that look linear looks linear, and its matrix is the product. Switch to the other pairs of maps and move the point around. The matrices change from point to point; the product rule between them does not.

The idea, in words

In one variable the proof of 1.4 runs: near $a$ , $g$ is a line; near $g(a)$ , $f$ is a line; so $f \circ g$ is a line of slope $f'(g(a))\,g'(a)$ , with an error that is still small. In many variables nothing changes except the meaning of “line”. Near $a$ the field $g$ is a constant plus a linear map plus a small error (the definition of differentiable, 5.3); near $b = g(a)$ the field $f$ is the same. Substitute the first into the second. A linear map applied to a linear map is a linear map (the product of the matrices). A linear map applied to a small error is a small error. A small error inside an error that is small compared with its input stays small.

Why· Why the outer Jacobian sits on the left and is evaluated at g(a)

Follow a tiny step $y$ at $a$ . The map $g$ turns it into the step $Dg(a)\,y$ at the point $b = g(a)$ . That step now lives where $f$ is being looked at, so $f$ turns it into $Df(b)\,(Dg(a)\,y)$ . The matrix that acts first is on the right, the one that acts second is on the left, and the second one is the derivative of $f$ at the place where the step lands, which is $b$ .

Where this came from· The curly d

The symbol $\partial$ for a partial derivative is older than its use in this chapter’s formulas. Condorcet used it in 1770, for partial differences. Legendre created the modern partial-derivative notation in 1786 and then dropped it, and Jacobi revived it in 1841, in the paper whose determinant appears in 5.3. The multivariable chain rule is usually written $\frac{\partial h}{\partial s} = \frac{\partial f}{\partial x}\frac{\partial x}{\partial s} + \frac{\partial f}{\partial y}\frac{\partial y}{\partial s}$ , which looks like cancelling fractions as in Leibniz’s one-variable $\frac{dz}{dx} = \frac{dz}{dy}\frac{dy}{dx}$ . The cancelling picture is a good memory aid and a bad proof: the sum over the routes from $s$ to $h$ is the matrix product, and the proof below does not cancel anything. The trap section shows where the aid fails.

The chain rule

Theorem T Apostol Thm 8.11 (8.20); Spagnolie Lecture 23

The chain rule for fields

Let $g$ map a set in $\mathbb R^p$ into $\mathbb R^n$ and $f$ map a set in $\mathbb R^n$ into $\mathbb R^m$ , and suppose $h = f \circ g$ is defined on a ball around $a$ . If $g$ is differentiable at $a$ with total derivative $g'(a)$ , and $f$ is differentiable at $b = g(a)$ with total derivative $f'(b)$ , then $h$ is differentiable at $a$ and its total derivative is the composition of linear maps

h'(a) = f'(b) \circ g'(a).

In Jacobian matrices, $Dh(a) = Df(b)\,Dg(a)$ , an $m \times p$ matrix equal to an $m \times n$ matrix times an $n \times p$ matrix. Entry by entry,

D_jh_i(a) = \sum_{k=1}^n D_kf_i(b)\,D_jg_k(a), \qquad i = 1, \dots, m,\quad j = 1, \dots, p.

In one lineIf $g$ is differentiable at $a$ and $f$ is differentiable at $b = g(a)$ , then $h = f \circ g$ is differentiable at $a$ and $h'(a) = f'(b) \circ g'(a)$ ; in matrices, $Dh(a) = Df(b)\,Dg(a)$ .

Why it is trueIt makes the derivative of any composition computable from the derivatives of the pieces, and it turns 'compose the maps' into 'multiply the matrices', which is the reason matrices were the right tool.

ProofApostol 8.20; Spagnolie Lecture 23 ('Read it!')

The idea: Write the change in g as a linear part plus a small error, and feed that change into the first-order formula for f. The linear parts compose to the product of the matrices. The leftover is two terms, each small compared with the step, because linear maps are continuous and because the change in g is no more than a constant times the step.

Let

v = g(a + y) - g(a)

. Then

v = Dg(a)\,y + \lVert y \rVert\,E_g(a, y)

with

E_g(a, y) \to 0

y \to 0

Name the small step

y

a

and the resulting step

v

b

;

g

is differentiable at

a

(definition in 5.3).

f(b + v) - f(b) = Df(b)\,v + \lVert v \rVert\,E_f(b, v)

with

E_f(b, v) \to 0

v \to 0

, and

E_f(b, O) = O

f

is differentiable at

b

. Differentiable fields are continuous (5.3), so

v \to O

y \to 0

and

b + v

stays inside the ball where the formula holds. We set

E_f(b, O) = O

because

v

can be

O

even when

y \ne O

(if

g

happens to take the same value twice), and then the term

\lVert v \rVert E_f

0

whatever

E_f

is.This is where the hypothesis 'h is defined near a' is used: we need b + v in the domain of f.

h(a + y) - h(a) = Df(b)\,Dg(a)\,y + \lVert y \rVert\,Df(b)\,E_g(a, y) + \lVert v \rVert\,E_f(b, v)

h(a + y) - h(a) = f(g(a + y)) - f(g(a)) = f(b + v) - f(b)

; substitute step 1 into the linear term of step 2 and use linearity of

Df(b)

h(a + y) - h(a) = Df(b)\,Dg(a)\,y + \lVert y \rVert\,E(a, y)

, where

E(a, y) = Df(b)\,E_g(a, y) + \dfrac{\lVert v \rVert}{\lVert y \rVert}\,E_f(b, v)

for

y \ne O

, and

E(a, O) = O

Pull out

\lVert y \rVert

from the last two terms. This is the form of the definition in 5.3 with the linear map

Df(b)\,Dg(a)

, which is linear because a product of matrices is a matrix.

The first term of

E

tends to

O

y \to 0

A linear map is continuous at

O

\lVert Df(b)\,w \rVert \le K\lVert w \rVert

for a constant

K

(the bound used in the proof of 'Differentiable fields are continuous', 5.3), and

E_g \to O

The quotient

\lVert v \rVert / \lVert y \rVert

is at most

M + 1

, a constant.

From step 1 and the triangle inequality,

\lVert v \rVert \le \lVert Dg(a)\,y \rVert + \lVert y \rVert\,\lVert E_g \rVert \le (M + \lVert E_g \rVert)\lVert y \rVert

. Here

M

is a constant with

\lVert Dg(a)\,w \rVert \le M\lVert w \rVert

for every

w

(a linear map has one, as in step 5), and

\lVert E_g \rVert \le 1

for small

y

because

E_g \to O

.The danger is the quotient: dividing by the small number ‖y‖. Step 1 shows v is itself no bigger than a constant times ‖y‖, so the quotient cannot blow up.

So the second term of

E

tends to

O

as well.

v \to O

y \to 0

(step 2), so

E_f(b, v) \to O

; a bounded number times something tending to

O

tends to

O

Hence

E(a, y) \to O

, and

Dh(a) = Df(b)\,Dg(a)

\blacksquare

Both terms of

E(a, y)

tend to

O

, so

h

satisfies the definition of differentiable with total derivative

Df(b)\,Dg(a)

. The entrywise formula is the definition of a matrix product.

Proof workshopRebuild the proof of the chain ruleApostol 8.20

Tap the lines in the order a proof would write them. Each line can only use what is already above it.

Quick checkIn

h = f \circ g

, at which point is the Jacobian

Df

evaluated in the chain rule?

Special cases

Every formula in the lectures is the matrix product for a particular choice of the shapes $p$ , $n$ , $m$ .

Inner field $g$	Outer field $f$	Shapes	The chain rule says
a path $r: \mathbb R \to \mathbb R^n$	scalar field	$(1 \times n)(n \times 1)$	$\dfrac{d}{dt}f(r(t)) = \nabla f(r(t))\cdot r'(t)$
$(s, t) \mapsto (x, y)$ , a change of coordinates	scalar field of $x, y$	$(1 \times 2)(2 \times 2)$	$h_s = f_x\,x_s + f_y\,y_s$ , $h_t = f_x\,x_t + f_y\,y_t$
a field $g: \mathbb R^p \to \mathbb R^n$	scalar field	$(1 \times n)(n \times p)$	$\nabla h(a)^{\mathsf T} = \nabla f(b)^{\mathsf T}\,Dg(a)$
a scalar field $g$	a function $F$ of one variable	$(1 \times 1)(1 \times n)$	$\nabla(F \circ g) = F'(g)\,\nabla g$
$g: \mathbb R^p \to \mathbb R^n$	$f: \mathbb R^n \to \mathbb R^m$	$(m \times n)(n \times p)$	$Dh = Df\,Dg$

The first row is the theorem “Rate of change along a curve” of 5.3, now a consequence of this chapter’s theorem: a row times a column is a dot product. The third row, transposed, is the form that backpropagation uses.

Corollary A Apostol 8.22 #12

The gradient of a composite

Under the hypotheses of the chain rule with $m = 1$ , $\nabla h(a) = \sum_{k} D_kf(b)\,\nabla g_k(a)$ . In matrix form $\nabla h(a) = Dg(a)^{\mathsf T}\,\nabla f(b)$ .

In one lineIf $f$ is a scalar field and $g = (g_1, \dots, g_n)$ a vector field, then $\nabla(f \circ g)(a) = \sum_{k=1}^n D_kf(g(a))\,\nabla g_k(a) = Dg(a)^{\mathsf T}\,\nabla f(g(a))$ .

Why it is trueIt says the gradient of the composite is a combination of the gradients of the pieces of $g$ , weighted by the partials of $f$ , and it is the vector-times-Jacobian-transpose form that reverse-mode differentiation computes.

ProofApostol 8.22 #12

The idea: Take the transpose of the matrix equation Dh = Df Dg.

\nabla h(a)^{\mathsf T} = \nabla f(b)^{\mathsf T}\,Dg(a)

The chain rule with

m = 1

Dh(a)

is the row

\nabla h(a)^{\mathsf T}

and

Df(b)

is the row

\nabla f(b)^{\mathsf T}

\nabla h(a) = Dg(a)^{\mathsf T}\,\nabla f(b)

Transpose of a product reverses the order (2.5).

Reading the product column by column gives

\sum_k D_kf(b)\,\nabla g_k(a)

The

j

-th entry of

Dg^{\mathsf T}\nabla f

\sum_k D_jg_k\,D_kf

; so the vector is

\sum_k D_kf(b)

times the

k

-th row of

Dg

, which is

\nabla g_k(a)

\blacksquare

Worked example

Differentiate

f(x, y, z) = x^2 + y^2 + z^2

along the helix

r(t) = (\cos t, \sin t, t)

by the chain rule for a path, and check by composing first.

Solution.

Identify the pieces. $f(x, y, z) = x^2 + y^2 + z^2$ , so $\nabla f = (2x, 2y, 2z)$ . The path is $r(t) = (\cos t, \sin t, t)$ with $r'(t) = (-\sin t, \cos t, 1)$ . The inner field has one input and three outputs, so $Dg$ is $3 \times 1$ and $Df$ is $1 \times 3$ .
Evaluate $\nabla f$ at $r(t)$ , not at $t$ . $\nabla f(r(t)) = (2\cos t, 2\sin t, 2t)$ . The outer derivative is evaluated at the image point.
Multiply. $\dfrac{d}{dt}f(r(t)) = 2\cos t(-\sin t) + 2\sin t\cos t + 2t\cdot1 = 2t$ . A row times a column is a dot product.
Check by composing first. $f(r(t)) = \cos^2t + \sin^2t + t^2 = 1 + t^2$ , with derivative $2t$ . Same answer.

Answer. $2t$ . The first two terms cancel because the point moves around a circle, at constant distance from the $z$ -axis; only the climb in $z$ changes $\lVert r \rVert^2$ .

Worked example Apostol 8.22 #14

Apostol 8.22 #14. Let

f(x, y) = (e^{x + 2y},\ \sin(y + 2x))

and

g(u, v, w) = (u + 2v^2 + 3w^3,\ 2v - u^2)

. Find

Df

Dg

, and

Dh(1, -1, 1)

for

h = f \circ g

Solution.

Jacobian of $f$ ( $2 \times 2$ ). $Df(x, y) = \begin{bmatrix} e^{x + 2y} & 2e^{x + 2y}\\ 2\cos(y + 2x) & \cos(y + 2x)\end{bmatrix}$ . Row 1 is the gradient of $e^{x + 2y}$ , row 2 the gradient of $\sin(y + 2x)$ .
Jacobian of $g$ ( $2 \times 3$ ). $Dg(u, v, w) = \begin{bmatrix} 1 & 4v & 9w^2\\ -2u & 2 & 0\end{bmatrix}$ . Two outputs, three inputs.
Where does $a = (1, -1, 1)$ go? $g(1, -1, 1) = (1 + 2 + 3,\ -2 - 1) = (6, -3)$ . This is the point $b$ at which $Df$ must be evaluated.
Evaluate both matrices. At $b = (6, -3)$ : $x + 2y = 0$ and $y + 2x = 9$ , so $Df(b) = \begin{bmatrix} 1 & 2\\ 2\cos9 & \cos9\end{bmatrix}$ . At $a$ : $Dg(a) = \begin{bmatrix} 1 & -4 & 9\\ -2 & 2 & 0\end{bmatrix}$ . Every entry is now a number.
Multiply, outer first. $Dh(a) = \begin{bmatrix} 1 & 2\\ 2\cos9 & \cos9\end{bmatrix}\begin{bmatrix} 1 & -4 & 9\\ -2 & 2 & 0\end{bmatrix} = \begin{bmatrix} -3 & 0 & 9\\ 0 & -6\cos9 & 18\cos9\end{bmatrix}$ . $(2 \times 2)(2 \times 3) = 2 \times 3$ . For example the $(2, 2)$ entry is $2\cos9\cdot(-4) + \cos9\cdot2 = -6\cos9$ .

Answer. $Dh(1, -1, 1) = \begin{bmatrix} -3 & 0 & 9\\ 0 & -6\cos9 & 18\cos9\end{bmatrix}$ . (Computing $h$ explicitly and differentiating gives the same matrix; the product is far shorter.)

Your answerIn the example above, what is the entry in row 2, column 3 of

Dh(1, -1, 1)

? (Give a decimal.)

Worked example Apostol 8.22 #15

Apostol 8.22 #15. Let

f(x, y, z) = (x^2 + y + z,\ 2x + y + z^2)

and

g(u, v, w) = (uv^2w^2,\ w^2\sin v,\ u^2e^v)

. Find

Dh(u, 0, w)

for

h = f \circ g

Solution.

Jacobians. $Df = \begin{bmatrix} 2x & 1 & 1\\ 2 & 1 & 2z\end{bmatrix}$ ( $2 \times 3$ ) and $Dg = \begin{bmatrix} v^2w^2 & 2uvw^2 & 2uv^2w\\ 0 & w^2\cos v & 2w\sin v\\ 2ue^v & u^2e^v & 0\end{bmatrix}$ ( $3 \times 3$ ). $f: \mathbb R^3 \to \mathbb R^2$ and $g: \mathbb R^3 \to \mathbb R^3$ .
Set $v = 0$ in $g$ first. $g(u, 0, w) = (0, 0, u^2)$ . So $Df$ is needed at $(x, y, z) = (0, 0, u^2)$ . Evaluate the outer matrix at the image point.
The two numerical matrices. $Df(0, 0, u^2) = \begin{bmatrix} 0 & 1 & 1\\ 2 & 1 & 2u^2\end{bmatrix}$ and $Dg(u, 0, w) = \begin{bmatrix} 0 & 0 & 0\\ 0 & w^2 & 0\\ 2u & u^2 & 0\end{bmatrix}$ . $\cos0 = 1$ , $\sin0 = 0$ , $e^0 = 1$ .
Multiply. $Dh(u, 0, w) = \begin{bmatrix} 2u & u^2 + w^2 & 0\\ 4u^3 & 2u^4 + w^2 & 0\end{bmatrix}$ . For example the $(2, 2)$ entry is $2\cdot0 + 1\cdot w^2 + 2u^2\cdot u^2$ .

Answer. $Dh(u, 0, w) = \begin{bmatrix} 2u & u^2 + w^2 & 0\\ 4u^3 & 2u^4 + w^2 & 0\end{bmatrix}$ . The zero last column says $h$ does not change in the $w$ direction at $v = 0$ : in $g$ the $w$ derivative is a multiple of $\sin v$ and $v^2$ .

Changing variables

Worked example Lecture 23

Polar coordinates. Let

h(r, \theta) = f(r\cos\theta,\ r\sin\theta)

for a differentiable scalar field

f

. Express

h_r

and

h_\theta

f_x

and

f_y

, and then express

f_x

and

f_y

h_r

and

h_\theta

Solution.

The inner map and its Jacobian. $P(r, \theta) = (r\cos\theta, r\sin\theta)$ has $DP = \begin{bmatrix}\cos\theta & -r\sin\theta\\ \sin\theta & r\cos\theta\end{bmatrix}$ (computed in 5.3). $x_r = \cos\theta$ , $x_\theta = -r\sin\theta$ , $y_r = \sin\theta$ , $y_\theta = r\cos\theta$ .
Apply the chain rule. $\begin{bmatrix} h_r & h_\theta\end{bmatrix} = \begin{bmatrix} f_x & f_y\end{bmatrix}\begin{bmatrix}\cos\theta & -r\sin\theta\\ \sin\theta & r\cos\theta\end{bmatrix}$ , that is

h_r = \cos\theta\,f_x + \sin\theta\,f_y,\qquad h_\theta = -r\sin\theta\,f_x + r\cos\theta\,f_y.

Row times matrix. The partials $f_x$ , $f_y$ are evaluated at $(r\cos\theta, r\sin\theta)$ . 3. Meaning. $h_r$ is the rate of change of $f$ per unit of distance moving straight outward; $h_\theta/r$ is the rate per unit of distance moving round the circle. A step of angle $d\theta$ moves a point at radius $r$ a distance $r\,d\theta$ , which is the factor $r$ in $h_\theta$ . 4. Invert to get the other direction. $\begin{bmatrix} f_x & f_y\end{bmatrix} = \begin{bmatrix} h_r & h_\theta\end{bmatrix}(DP)^{-1}$ and $(DP)^{-1} = \dfrac1r\begin{bmatrix} r\cos\theta & r\sin\theta\\ -\sin\theta & \cos\theta\end{bmatrix}$ , so

f_x = \cos\theta\,h_r - \frac{\sin\theta}{r}\,h_\theta,\qquad f_y = \sin\theta\,h_r + \frac{\cos\theta}{r}\,h_\theta.

Multiply the chain rule on the right by the inverse matrix (2.6); it exists when $r \ne 0$ , since $\det DP = r$ . 5. Check on $f = x^2y$ . Then $h = r^3\cos^2\theta\sin\theta$ . At $r = 2$ , $\theta = \pi/6$ : $x = \sqrt3$ , $y = 1$ , $f_x = 2xy = 2\sqrt3$ , $f_y = x^2 = 3$ , so $h_\theta = -2\cdot\frac12\cdot2\sqrt3 + 2\cdot\frac{\sqrt3}2\cdot3 = \sqrt3$ . Differentiating $h$ directly: $h_\theta = r^3(\cos^3\theta - 2\cos\theta\sin^2\theta)$ , which is $8\cdot\big(\frac{3\sqrt3}8 - \frac{\sqrt3}4\big) = \sqrt3$ . Two routes, one number.

Answer. $h_r = \cos\theta\,f_x + \sin\theta\,f_y$ and $h_\theta = -r\sin\theta\,f_x + r\cos\theta\,f_y$ ; the reverse formulas are in step 4.

Your answerFor

f(x, y) = x^2y

and

h(r, \theta) = f(r\cos\theta, r\sin\theta)

, find

h_\theta

r = 2

\theta = \pi/6

. (Give a decimal.)

Worked example Practice Final 2013, problem 9

Practice Final 2013, problem 9. Let

w = f(u, v)

with

u = xy

and

v = x/y

. Using the chain rule, (a) find

w_x

and

w_y

; (b) find

w_{xx}

Solution.

The inner Jacobian. $(x, y) \mapsto (u, v) = (xy, x/y)$ has $u_x = y$ , $u_y = x$ , $v_x = 1/y$ , $v_y = -x/y^2$ . Ordinary partial derivatives of the two formulas.
(a) First derivatives. $w_x = f_uu_x + f_vv_x = y\,f_u + \dfrac1y\,f_v$ and $w_y = f_uu_y + f_vv_y = x\,f_u - \dfrac{x}{y^2}\,f_v$ . Row $(f_u, f_v)$ times the matrix.
(b) Differentiate $w_x$ in $x$ , by the product rule. In $w_x = y\,f_u + y^{-1}f_v$ the factors $y$ and $y^{-1}$ do not depend on $x$ , but $f_u$ and $f_v$ do, through $u$ and $v$ . This is the step students forget: $f_u(xy, x/y)$ is a composite field, so it needs its own chain rule.
Chain rule on $f_u$ and $f_v$ . $\partial_x(f_u) = f_{uu}u_x + f_{uv}v_x = y\,f_{uu} + \dfrac1y\,f_{uv}$ and $\partial_x(f_v) = y\,f_{uv} + \dfrac1y\,f_{vv}$ (using $f_{uv} = f_{vu}$ ). Same rule as step 2, with $f_u$ and $f_v$ in place of $f$ .
Assemble. $w_{xx} = y\Big(y\,f_{uu} + \dfrac1y\,f_{uv}\Big) + \dfrac1y\Big(y\,f_{uv} + \dfrac1y\,f_{vv}\Big) = y^2f_{uu} + 2f_{uv} + \dfrac1{y^2}f_{vv}$ . The two $f_{uv}$ terms add.

Answer. $w_x = yf_u + y^{-1}f_v$ , $w_y = xf_u - xy^{-2}f_v$ , $w_{xx} = y^2f_{uu} + 2f_{uv} + y^{-2}f_{vv}$ . Test with $f = u^2v$ : then $w = x^3y$ and $w_{xx} = 6xy$ , while the formula gives $y^2\cdot2v + 2\cdot2u + 0 = 2y^2\cdot\frac xy + 4xy = 6xy$ .

Your answerWith

w = f(u, v) = u^2v

u = xy

v = x/y

, compute

w_x

(x, y) = (2, 1)

by the chain rule formula

w_x = y f_u + f_v/y

Worked example MATH 375 Final Review, 8.21

Let

h(u, v) = f(x(u, v), y(u, v))

. Find the mixed second derivative

h_{uv} = \partial_v(h_u)

: differentiate in

u

first, then in

v

Solution.

First derivative in $u$ . $h_u = f_x\,x_u + f_y\,y_u$ . Chain rule; $f_x$ and $f_y$ are evaluated at $(x(u, v), y(u, v))$ .
Differentiate in $v$ , product rule on each term. $\partial_v(h_u) = \big(\partial_vf_x\big)x_u + f_x\,x_{uv} + \big(\partial_vf_y\big)y_u + f_y\,y_{uv}$ . $x_u$ is a function of $(u, v)$ too, so it is differentiated with the product rule.
Chain rule on the two partials. $\partial_vf_x = f_{xx}x_v + f_{xy}y_v$ and $\partial_vf_y = f_{yx}x_v + f_{yy}y_v$ . $f_x$ and $f_y$ are themselves composite fields, and each gets its own chain rule.
Assemble. $h_{uv} = \big[f_{xx}x_v + f_{xy}y_v\big]x_u + f_x\,x_{uv} + \big[f_{yx}x_v + f_{yy}y_v\big]y_u + f_y\,y_{uv}$ .

Answer. The formula in step 4, which is the one on the Final Review. With $f_{xy} = f_{yx}$ it collects as $f_{xx}x_ux_v + f_{xy}(x_uy_v + x_vy_u) + f_{yy}y_uy_v + f_xx_{uv} + f_yy_{uv}$ .

The inverse of a field has the inverse matrix as its derivative

Definition T

Jacobian determinant

When $m = n$ , the Jacobian matrix $J_F(a)$ is square and its determinant (2.7) is the Jacobian determinant $\det J_F(a) = \dfrac{\partial(F_1, \dots, F_n)}{\partial(x_1, \dots, x_n)}$ . A tiny cube of side $\epsilon$ at $a$ goes to a tiny parallelepiped of volume about $|\det J_F(a)|\,\epsilon^n$ . For polar coordinates it is $r$ (5.3).

In one lineFor a field $F: \mathbb R^n \to \mathbb R^n$ with the same number of inputs and outputs, $\det J_F(a)$ : the factor by which $F$ scales small volumes near $a$ .

Why it existsIt is the one number that records whether $F$ locally squashes space, flips it, or stretches it, and it is the correction factor in the change of variables formula for integrals.

Corollary T chain rule and the product rule for determinants, 2.7

Jacobian determinants multiply

For $h = f \circ g$ with $f, g: \mathbb R^n \to \mathbb R^n$ , $\det Dh(a) = \det Df(g(a))\cdot\det Dg(a)$ .

In one lineIf $f$ and $g$ both map $\mathbb R^n$ to $\mathbb R^n$ , then $\det D(f \circ g)(a) = \det Df(g(a))\cdot\det Dg(a)$ .

Why it is trueIt says volume scalings multiply along a composition, which is the geometric meaning of the chain rule for square Jacobians.

Proof2.7

The idea: Take the determinant of the matrix identity Dh = Df Dg and use that the determinant of a product is the product of the determinants.

Dh(a) = Df(g(a))\,Dg(a)

The chain rule.

\det Dh(a) = \det Df(g(a))\cdot\det Dg(a)

Product rule for determinants, valid since both matrices are

n \times n

\blacksquare

Definition B

Diffeomorphism

A diffeomorphism is a bijection $F: U \to V$ of open sets $U, V \subseteq \mathbb R^n$ such that $F$ and $F^{-1}$ are both differentiable. (Authors who want second derivatives require continuously differentiable, and the name is usually reserved for that stronger case. The proofs below use only differentiability.)

In one lineA differentiable field $F: U \to V$ between open sets of $\mathbb R^n$ that has a differentiable inverse $F^{-1}: V \to U$ .

Why it existsIt is a change of coordinates that can be undone without losing differentiability, which is the kind of map along which one may substitute variables freely.

Theorem T MATH 375 Practice Final 2013, problem 10; Apostol 8.22

The derivative of an inverse field

Let $F: U \to V$ be a bijection of open sets in $\mathbb R^n$ with $F$ differentiable at $x$ and $G = F^{-1}$ differentiable at $F(x)$ . Then $DF(x)$ is invertible and $DG(F(x)) = \big(DF(x)\big)^{-1}$ .

In one lineIf $F: U \to V$ is a diffeomorphism, then $DF(x)$ is an invertible matrix for every $x$ , and $D(F^{-1})(F(x)) = \big(DF(x)\big)^{-1}$ .

Why it is trueIt tells you the derivative of an inverse without finding the inverse: invert a matrix instead. It also shows that a differentiable inverse forces $\det DF \ne 0$ .

ProofPractice Final 2013, problem 10 (hint: apply the chain rule)

The idea: Apply the chain rule to the two identities G∘F = identity and F∘G = identity. The derivative of the identity is the identity matrix, so the two Jacobians multiply to I in both orders.

The composite

G \circ F

is the identity field on

U

, whose Jacobian is the identity matrix

I

(5.3: the derivative of

x \mapsto x

G(F(x)) = x

for

x \in U

DG(F(x))\,DF(x) = I

The chain rule, with the outer field

G

evaluated at the image point

F(x)

Likewise

DF(x)\,DG(F(x)) = I

F(G(y)) = y

for

y \in V

; the same two steps with the roles swapped, at

y = F(x)

where

G(y) = x

DF(x)

is invertible and

DG(F(x)) = DF(x)^{-1}

\blacksquare

A matrix with a two-sided inverse is invertible and the inverse is unique (2.4). Only one identity is needed for square matrices, since a left inverse of a square matrix is two-sided.

Quick check

F(x) = x^3

is a differentiable bijection of

\mathbb R

onto

\mathbb R

. What does the theorem say about its inverse

G(y) = y^{1/3}

y = 0

Worked example Practice Final 2013, problem 10; Apostol 8.17 #5

Practice Final 2013, problem 10, in numbers. The field

F(x, y) = (e^x\cos y,\ e^x\sin y)

is a diffeomorphism from the strip

0 < y < \pi

onto the upper half plane, with inverse

G(X, Y) = \big(\tfrac12\ln(X^2 + Y^2),\ \arg(X + iY)\big)

. Find

DG

F(\ln2, \pi/6)

without differentiating

G

Solution.

The image point. $F(\ln2, \pi/6) = (2\cos\frac\pi6,\ 2\sin\frac\pi6) = (\sqrt3, 1)$ . $e^{\ln2} = 2$ .
$DF$ at $(\ln2, \pi/6)$ . $DF = \begin{bmatrix} e^x\cos y & -e^x\sin y\\ e^x\sin y & e^x\cos y\end{bmatrix} = \begin{bmatrix}\sqrt3 & -1\\ 1 & \sqrt3\end{bmatrix}$ . $2\cdot\frac{\sqrt3}2$ and $2\cdot\frac12$ .
Invert it. The determinant is $3 + 1 = 4$ , so $(DF)^{-1} = \dfrac14\begin{bmatrix}\sqrt3 & 1\\ -1 & \sqrt3\end{bmatrix}$ . The $2 \times 2$ inverse swaps the diagonal, negates the off-diagonal, and divides by the determinant.
Check with the real inverse. $DG = \begin{bmatrix}\frac{X}{X^2 + Y^2} & \frac{Y}{X^2 + Y^2}\\ \frac{-Y}{X^2 + Y^2} & \frac{X}{X^2 + Y^2}\end{bmatrix}$ , which at $(\sqrt3, 1)$ (where $X^2 + Y^2 = 4$ ) is $\frac14\begin{bmatrix}\sqrt3 & 1\\ -1 & \sqrt3\end{bmatrix}$ . Differentiate $\frac12\ln(X^2 + Y^2)$ and $\arctan(Y/X)$ .

Answer. $DG(\sqrt3, 1) = \frac14\begin{bmatrix}\sqrt3 & 1\\ -1 & \sqrt3\end{bmatrix}$ . Because the rows of $DG$ are $\nabla u$ and $\nabla v$ for $u = \frac12\ln(X^2 + Y^2)$ and $v = \arg$ , and they are perpendicular, this is also Apostol 8.17 #5: the gradients of $u$ and $v$ are perpendicular at each point.

Your answerIn the example above, what is the entry in row 1, column 1 of

DG(\sqrt3, 1)

? (Give a decimal.)

Worked example Lecture 23, with the inverse theorem

Find

D(P^{-1})

for polar coordinates at the point

(x, y) = (1, 1)

by inverting

DP

, and compare with differentiating

r = \sqrt{x^2 + y^2}

\theta = \arctan(y/x)

Solution.

The point in polar coordinates. $(1, 1)$ is $r = \sqrt2$ , $\theta = \pi/4$ . Distance from the origin and angle.
Invert. $DP(\sqrt2, \tfrac\pi4) = \begin{bmatrix}\frac{\sqrt2}2 & -1\\ \frac{\sqrt2}2 & 1\end{bmatrix}$ has determinant $\sqrt2 = r$ , so its inverse is $\dfrac1{\sqrt2}\begin{bmatrix}1 & 1\\ -\frac{\sqrt2}2 & \frac{\sqrt2}2\end{bmatrix} = \begin{bmatrix}\frac{\sqrt2}2 & \frac{\sqrt2}2\\ -\frac12 & \frac12\end{bmatrix}$ . Same $2 \times 2$ inverse recipe.
Differentiate the inverse map. $r_x = x/r = \frac{\sqrt2}2$ , $r_y = \frac{\sqrt2}2$ , $\theta_x = -y/r^2 = -\frac12$ , $\theta_y = x/r^2 = \frac12$ . $\theta = \arctan(y/x)$ has $\theta_x = \frac{-y/x^2}{1 + y^2/x^2} = \frac{-y}{x^2 + y^2}$ .

Answer. $\begin{bmatrix}\frac{\sqrt2}2 & \frac{\sqrt2}2\\ -\frac12 & \frac12\end{bmatrix}$ both ways. The first row says $\nabla r$ is the unit vector $(1, 1)/\sqrt2$ pointing away from the origin, as in 5.2.

Beyond the course: for understanding· A preview: a nonzero Jacobian determinant is also enough, locally

The theorem above assumes an inverse and concludes $\det DF \ne 0$ . The inverse function theorem goes the other way: if $F$ is continuously differentiable near $a$ and $\det DF(a) \ne 0$ , then $F$ has a differentiable inverse on some small ball around $a$ . It is proved in 5.7 by a contraction argument (4.2). The example $F(x, y) = (e^x\cos y, e^x\sin y)$ shows why “locally” is the right word: $\det DF = e^{2x} \ne 0$ everywhere, but $F$ is not one-to-one on the whole plane, since $y$ and $y + 2\pi$ give the same point.

Second derivatives of a composite

The chain rule applied once gives first derivatives. The first partials of $f$ are again functions of position, so inside a composite they are composites, and the rule applies to them. The whole difficulty is bookkeeping: a product rule at every stage. Two sentences carry it: differentiate; then differentiate each partial of $f$ that appears, by the chain rule again.

Theorem T Apostol 8.17 #1; Spagnolie Lecture 23; Fall 2025 final problem 5

Second derivative along a path

Let $r: (\alpha, \beta) \to \mathbb R^n$ be twice differentiable, $f$ a scalar field whose first partials are differentiable (continuous second partials suffice), and $g(t) = f(r(t))$ . Then

g'(t) = \sum_i D_if(r(t))\,r_i'(t), \qquad g''(t) = \sum_{i=1}^n\sum_{j=1}^n D_jD_if(r(t))\,r_i'(t)\,r_j'(t) + \sum_{i=1}^n D_if(r(t))\,r_i''(t).

With the matrix $H = [D_jD_if]$ of second partials, this reads $g'' = r'^{\mathsf T}H\,r' + \nabla f\cdot r''$ .

In one lineIf $r$ is twice differentiable and $f$ has continuous second partials, then $\dfrac{d^2}{dt^2}f(r(t)) = \sum_{i,j} D_jD_if(r(t))\,r_i'(t)\,r_j'(t) + \nabla f(r(t))\cdot r''(t)$ .

Why it is trueIt splits the acceleration of the value of $f$ into the part due to the curvature of $f$ (the quadratic form of the matrix of second partials, which 5.6 calls the Hessian) and the part due to the acceleration of the path.

ProofApostol 8.17 #1(b)

The idea: The first derivative is a sum of products of two functions of t. Differentiate with the product rule. One factor is r_i′, giving r_i″. The other factor is D_i f evaluated along the path, which is a composite field, so apply the first chain rule to it.

g'(t) = \sum_i D_if(r(t))\,r_i'(t)

The chain rule for a path.

g''(t) = \sum_i\Big[\dfrac{d}{dt}D_if(r(t))\Big]r_i'(t) + \sum_iD_if(r(t))\,r_i''(t)

Product rule on each term of the sum.

Substitute for the bracket.

D_if

is a differentiable scalar field, so the chain rule for a path applies to it:

\frac{d}{dt}D_if(r(t)) = \sum_jD_jD_if(r(t))\,r_j'(t)

. This is the step the homework warned about.Skipping it is the standard error: f_x(X(t), Y(t)) depends on t, even though f_x is a partial derivative.

g'' = \sum_{i,j}D_jD_if\,r_i'r_j' + \sum_iD_if\,r_i''

The result is the formula claimed; the quadratic form of

H

is the same whether

H

or its transpose is used, so the order of the indices does not matter in the double sum.

\blacksquare

Worked example MATH 375 Fall 2025 final, problem 5

Fall 2025 final, problem 5. A buoy at

(X(t), Y(t))

measures a temperature

f(x, y, t)

that also changes in time. Let

g(t) = f(X(t), Y(t), t)

. Find

g'

and

g''

Solution.

The path in space-time. $r(t) = (X(t), Y(t), t)$ , so $r' = (X', Y', 1)$ and $r'' = (X'', Y'', 0)$ . Time is the third coordinate; its velocity is $1$ and its acceleration $0$ .
First derivative. $g' = f_xX' + f_yY' + f_t$ , with the partials evaluated at $(X(t), Y(t), t)$ . Already in 5.3.
Differentiate each term, product rule. $g'' = \big(\tfrac{d}{dt}f_x\big)X' + f_xX'' + \big(\tfrac{d}{dt}f_y\big)Y' + f_yY'' + \tfrac{d}{dt}f_t$ . The factor $1$ in front of $f_t$ is constant.
Chain rule on each partial. $\tfrac{d}{dt}f_x = f_{xx}X' + f_{xy}Y' + f_{xt}$ , $\tfrac{d}{dt}f_y = f_{yx}X' + f_{yy}Y' + f_{yt}$ , $\tfrac{d}{dt}f_t = f_{tx}X' + f_{ty}Y' + f_{tt}$ . The velocity of the path has three entries, so each partial has three terms. The last entry $1$ is why the lone $f_{xt}$ , $f_{yt}$ , $f_{tt}$ appear.
Collect, assuming equal mixed partials. $g'' = f_{xx}X'^2 + f_{yy}Y'^2 + f_{tt} + 2f_{xy}X'Y' + 2f_{xt}X' + 2f_{yt}Y' + f_xX'' + f_yY''$ .

Answer. The formula in step 5. For the final’s example $f = xt$ , $X = t$ , $Y = 0$ : $f_{xt} = 1$ and all other second partials vanish, $g = t^2$ , so $g'' = 2f_{xt}X' = 2$ , which is $\frac{d^2}{dt^2}t^2$ .

Worked example Lecture 23 (left unfinished on the page); Apostol 8.22 #5

Lecture 23, finished. For

h(r, \theta) = f(r\cos\theta, r\sin\theta)

find

h_{\theta\theta}

h_{rr}

and

h_{r\theta}

, and deduce the Laplacian

f_{xx} + f_{yy}

in polar coordinates.

Solution.

Start from $h_\theta$ . $h_\theta = -r\sin\theta\,f_x + r\cos\theta\,f_y$ (previous example). Differentiate this in $\theta$ , with $r$ held fixed.
Product rule on each term. $h_{\theta\theta} = -r\cos\theta\,f_x - r\sin\theta\,\partial_\theta(f_x) - r\sin\theta\,f_y + r\cos\theta\,\partial_\theta(f_y)$ . The coefficients depend on $\theta$ and so do $f_x$ , $f_y$ . The two terms $-r\cos\theta\,f_x$ and $-r\sin\theta\,f_y$ carry minus signs; the unfinished page of Lecture 23 had the opposite overall sign on this line.
Chain rule on $f_x$ and $f_y$ , with $\partial_\theta(x) = -r\sin\theta$ and $\partial_\theta(y) = r\cos\theta$ . $\partial_\theta(f_x) = -r\sin\theta\,f_{xx} + r\cos\theta\,f_{xy}$ and $\partial_\theta(f_y) = -r\sin\theta\,f_{xy} + r\cos\theta\,f_{yy}$ . Same rule as for $h_\theta$ with $f_x$ in place of $f$ .
Collect. $h_{\theta\theta} = -r\cos\theta\,f_x - r\sin\theta\,f_y + r^2\sin^2\theta\,f_{xx} - 2r^2\sin\theta\cos\theta\,f_{xy} + r^2\cos^2\theta\,f_{yy}$ . The two cross terms $r^2\sin\theta\cos\theta\,f_{xy}$ each come with a minus sign and add.
The other two. Differentiating $h_r = \cos\theta\,f_x + \sin\theta\,f_y$ in $r$ (the coefficients do not depend on $r$ ): $h_{rr} = \cos^2\theta\,f_{xx} + 2\sin\theta\cos\theta\,f_{xy} + \sin^2\theta\,f_{yy}$ . Differentiating it in $\theta$ : $h_{r\theta} = -\sin\theta\,f_x + \cos\theta\,f_y + r\big(-\sin\theta\cos\theta\,f_{xx} + (\cos^2\theta - \sin^2\theta)f_{xy} + \sin\theta\cos\theta\,f_{yy}\big)$ . Same two sentences, applied to $h_r$ .
Add $h_{rr}$ , $\frac1rh_r$ and $\frac1{r^2}h_{\theta\theta}$ . The $f_{xy}$ terms cancel, the $f_{xx}$ terms give $\cos^2\theta + \sin^2\theta = 1$ , likewise for $f_{yy}$ , and the first-order terms give $\frac1r(\cos\theta f_x + \sin\theta f_y) - \frac1r(\cos\theta f_x + \sin\theta f_y) = 0$ . Check the $f_x$ terms: from $\frac1rh_r$ we get $\frac{\cos\theta}rf_x$ , from $\frac1{r^2}h_{\theta\theta}$ we get $-\frac{\cos\theta}rf_x$ .

Answer. $h_{\theta\theta}$ as in step 4, and

f_{xx} + f_{yy} = h_{rr} + \frac1r\,h_r + \frac1{r^2}\,h_{\theta\theta}.

Check with $f = x^2 - y^2$ , so $h = r^2\cos2\theta$ : $2\cos2\theta + 2\cos2\theta - 4\cos2\theta = 0$ , and $f_{xx} + f_{yy} = 2 - 2 = 0$ .

Theorem B the compact form of Apostol 8.22 #3 and 8.24 #7

Second derivatives of a composite field

Let $g: \mathbb R^p \to \mathbb R^n$ and $f: \mathbb R^n \to \mathbb R$ have continuous second partials and $h = f \circ g$ , $b = g(a)$ . Write $H_f(b) = [D_lD_kf(b)]_{k,l}$ , an $n \times n$ matrix, and $H_{g_k}(a) = [D_jD_ig_k(a)]_{i,j}$ , a $p \times p$ matrix. Then the $p \times p$ matrix of second partials of $h$ at $a$ is

H_h(a) = Dg(a)^{\mathsf T}\,H_f(b)\,Dg(a) + \sum_{k=1}^n D_kf(b)\,H_{g_k}(a).

In one lineIf $h = f \circ g$ with $f$ a scalar field and $g: \mathbb R^p \to \mathbb R^n$ , all $C^2$ , then the matrix of second partials of $h$ is $Dg(a)^{\mathsf T}\,[D_lD_kf(b)]\,Dg(a) + \sum_{k=1}^n D_kf(b)\,[D_jD_ig_k(a)]$ .

Why it is trueIt packages every polar, cylindrical and homework computation into one matrix identity, and it shows what happens at a critical point of $f$ (the second sum vanishes), which is the reason the type of a critical point does not depend on the coordinates (5.6).

Proofthe proof of the previous theorem, entrywise

The idea: Differentiate the entrywise first-derivative formula once more: the product rule splits each term into a part where the partial of f is differentiated and a part where the partial of g is.

D_ih(a) = \sum_kD_kf(g(a))\,D_ig_k(a)

Chain rule, entrywise.

D_jD_ih = \sum_k\big[D_j\big(D_kf\circ g\big)\big]D_ig_k + \sum_kD_kf(g)\,D_jD_ig_k

Product rule in the variable

x_j

D_jD_ih = \sum_{k,l}D_lD_kf(g)\,D_jg_l\,D_ig_k + \sum_kD_kf(g)\,D_jD_ig_k

D_kf

is a differentiable scalar field; chain rule:

D_j(D_kf\circ g) = \sum_lD_lD_kf(g)\,D_jg_l

Written as matrices this is the claimed identity.

The first double sum is entry

(i, j)

Dg^{\mathsf T}H_fDg

, since

D_ig_k

is entry

(k, i)

Dg

and

D_jg_l

is entry

(l, j)

\blacksquare

Worked example Apostol 8.24 #6

Apostol 8.24 #6. The change of variables

x = u + v

y = uv^2

transforms

f(x, y)

into

g(u, v)

. Find

\dfrac{\partial^2g}{\partial v\,\partial u}

u = v = 1

, given that at the corresponding point

f_y = f_{xx} = f_{yy} = f_{xy} = f_{yx} = 1

Solution.

The point and the partials of the change. $u = v = 1$ gives $(x, y) = (2, 1)$ . $x_u = 1$ , $x_v = 1$ , $y_u = v^2$ , $y_v = 2uv$ . Differentiate $x = u + v$ and $y = uv^2$ .
First derivative in $u$ . $g_u = f_x\cdot1 + f_y\,v^2$ . Chain rule.
Differentiate in $v$ , product rule on the second term. $\partial_v(g_u) = \partial_v(f_x) + 2v\,f_y + v^2\,\partial_v(f_y)$ . $v^2$ is a function of $v$ as well.
Chain rule on $f_x$ and $f_y$ (the $v$ -derivatives). $\partial_vf_x = f_{xx}x_v + f_{xy}y_v = f_{xx} + 2uv\,f_{xy}$ and $\partial_vf_y = f_{yx} + 2uv\,f_{yy}$ . $x_v = 1$ and $y_v = 2uv$ .
Substitute the numbers at $u = v = 1$ . $\partial_v(g_u) = (1 + 2\cdot1) + 2\cdot1 + 1\cdot(1 + 2\cdot1) = 3 + 2 + 3 = 8$ .

Answer. $8$ .

Your answerIn the example above, what is

\partial^2g/\partial v\,\partial u

u = v = 1

Worked example Apostol 8.22 #3 and #4(a) (MATH 375 HW 13)

Homework 13, Apostol 8.22 #3 and #4(a). The equations

x = X(s, t)

y = Y(s, t)

turn

f(x, y)

into

F(s, t)

. Find

F_s

F_{ss}

F_{st}

F_{tt}

in general, then for

X = s + t

Y = st

Solution.

First derivatives. $F_s = f_x\,X_s + f_y\,Y_s$ and $F_t = f_x\,X_t + f_y\,Y_t$ . The chain rule with the row $(f_x, f_y)$ times the matrix with columns $(X_s, Y_s)$ and $(X_t, Y_t)$ .
Differentiate $F_s$ in $s$ , product rule on each term. $F_{ss} = \big(\partial_sf_x\big)X_s + f_x\,X_{ss} + \big(\partial_sf_y\big)Y_s + f_y\,Y_{ss}$ . $X_s$ and $Y_s$ are functions of $(s, t)$ too.
Chain rule on $f_x$ and $f_y$ . $\partial_sf_x = f_{xx}X_s + f_{xy}Y_s$ and $\partial_sf_y = f_{yx}X_s + f_{yy}Y_s$ . Each partial of $f$ is a composite field, as in the previous examples.
Collect, with $f_{xy} = f_{yx}$ . $F_{ss} = f_{xx}X_s^2 + 2f_{xy}X_sY_s + f_{yy}Y_s^2 + f_x\,X_{ss} + f_y\,Y_{ss}$ . The same steps, differentiating $F_s$ in $t$ and $F_t$ in $t$ , give $F_{st} = f_{xx}X_sX_t + f_{xy}(X_sY_t + X_tY_s) + f_{yy}Y_sY_t + f_x\,X_{st} + f_y\,Y_{st}$ and $F_{tt} = f_{xx}X_t^2 + 2f_{xy}X_tY_t + f_{yy}Y_t^2 + f_x\,X_{tt} + f_y\,Y_{tt}$ . Compare with the matrix identity of the theorem above: the quadratic parts are $Dg^{\mathsf T}H_fDg$ , the rest is the sum with the second derivatives of $X$ and $Y$ .
Insert $X = s + t$ , $Y = st$ . Then $X_s = X_t = 1$ , $Y_s = t$ , $Y_t = s$ , all second derivatives of $X$ vanish, and $Y_{ss} = Y_{tt} = 0$ , $Y_{st} = 1$ . Differentiate the two formulas.
Read off. $F_s = f_x + t\,f_y$ , $F_t = f_x + s\,f_y$ , $F_{ss} = f_{xx} + 2t\,f_{xy} + t^2f_{yy}$ , $F_{tt} = f_{xx} + 2s\,f_{xy} + s^2f_{yy}$ , $F_{st} = f_{xx} + (s + t)f_{xy} + st\,f_{yy} + f_y$ . The only first-order term that survives is $f_y\,Y_{st}$ . Test with $f = xy$ : $F = (s + t)st$ , $F_{st} = 2s + 2t$ , and the formula gives $0 + (s + t)\cdot1 + 0 + f_y = (s + t) + (s + t)$ , since $f_y = x = s + t$ .

Answer. The general formulas of step 4, and the special case of step 6.

Worked example Apostol 8.9 #3 (MATH 375 HW 11 style)

Apostol 8.9 #3. For a linear map

T: \mathbb R^n \to \mathbb R^n

and the scalar field

f(x) = x\cdot T(x)

, compute

f'(x; y)

and

\nabla f

Solution.

Write $f$ as a composite. $f = m \circ G$ where $G(x) = (x, T(x))$ goes from $\mathbb R^n$ to $\mathbb R^n \times \mathbb R^n$ and $m(u, w) = u\cdot w$ is the dot product. The pieces are simple: $G$ is linear and $m$ is bilinear.
Derivative of the dot product. $m(u + p, w + q) = u\cdot w + (p\cdot w + u\cdot q) + p\cdot q$ , so $Dm(u, w)(p, q) = p\cdot w + u\cdot q$ . As for $xy$ in 5.3: the cross term $p\cdot q$ is second order.
Chain rule. $f'(x; y) = Dm(G(x))(DG(x)\,y) = Dm(x, Tx)(y, Ty) = y\cdot T(x) + x\cdot T(y)$ . $DG(x)\,y = (y, T(y))$ because $G$ is linear.
The gradient. If $T$ has the matrix $A$ , then $f(x) = x^{\mathsf T}Ax$ and $f'(x; y) = y^{\mathsf T}Ax + x^{\mathsf T}Ay = y\cdot(A + A^{\mathsf T})x$ , so $\nabla f = (A + A^{\mathsf T})x$ . Compare 3.8: the rule for $\nabla(x^{\mathsf T}Ax)$ , now proved from the chain rule instead of by expanding sums.

Answer. $f'(x; y) = x\cdot T(y) + y\cdot T(x)$ , and $\nabla f = (A + A^{\mathsf T})x$ , which is $2Ax$ when $T$ is symmetric.

Quick checkFor

g(t) = f(X(t), Y(t))

, what is

\dfrac{d}{dt}\big[f_x(X(t), Y(t))\big]

Quick checkIn the polar formula for

h_{\theta\theta}

there are terms

-r\cos\theta\,f_x - r\sin\theta\,f_y

with no second derivative. Where do they come from?

Proof workshopGive the reason for each line of the second derivative along a pathApostol 8.17 #1

Each line of the proof is shown in order. Choose the reason that justifies it, then press Check.

$g'(t) = \sum_i D_i f(r(t))\,r_i'(t)$ .
$g''(t) = \sum_i \big[\tfrac{d}{dt} D_i f(r(t))\big] r_i'(t) + \sum_i D_i f(r(t))\, r_i''(t)$ .
$\tfrac{d}{dt} D_i f(r(t)) = \sum_j D_j D_i f(r(t))\, r_j'(t)$ .
$g'' = \sum_{i,j} D_j D_i f\; r_i' r_j' + \sum_i D_i f\; r_i''$ .

Where it is used

Used inside maths· Inside maths: changing variables to solve a PDE, and the Laplacian in polar coordinates

The wave equation $u_{tt} = c^2u_{xx}$ describes a vibrating string. Put $\xi = x - ct$ , $\eta = x + ct$ and write $u(x, t) = U(\xi, \eta)$ . By the second-derivative chain rule (the example with $\xi_x = \eta_x = 1$ , $\xi_t = -c$ , $\eta_t = c$ ): $u_{xx} = U_{\xi\xi} + 2U_{\xi\eta} + U_{\eta\eta}$ and $u_{tt} = c^2\big(U_{\xi\xi} - 2U_{\xi\eta} + U_{\eta\eta}\big)$ , so

u_{tt} - c^2u_{xx} = -4c^2\,U_{\xi\eta}.

The equation becomes $U_{\xi\eta} = 0$ , whose solutions are $U = F(\xi) + G(\eta)$ (integrate in $\eta$ , then in $\xi$ ; 5.5 says when this is legitimate). That is d’Alembert’s solution $u = F(x - ct) + G(x + ct)$ : a wave of any shape moving right plus a wave moving left. The polar Laplacian of the example above does the same job for Laplace’s equation on a disc: $f_{xx} + f_{yy} = 0$ becomes $r^2h_{rr} + rh_r + h_{\theta\theta} = 0$ , which separates into one-variable equations and has solutions $r^n\cos n\theta$ and $r^n\sin n\theta$ , the real parts and imaginary parts of $z^n$ from 0.6. The chain rule turns a problem about a disc into a problem about a rectangle.

Used in the real world· AI: backpropagation is this chapter's product, multiplied from the left

Take the small network of 3.8 with a vector input: $x = (1, 2)$ , a weight matrix $W = \begin{bmatrix}1 & -1\\ 2 & 1\end{bmatrix}$ , $z = Wx = (-1, 4)$ , a ReLU $h = \max(0, z) = (0, 4)$ , an output $y = v\cdot h$ with $v = (1, -1)$ , so $y = -4$ , and a loss $\ell = \frac12(y - 1)^2 = 12.5$ . The loss is a composite of four fields, so its derivative with respect to the input is a product of four Jacobians:

D_x\ell = \underbrace{(y - 1)}_{1 \times 1}\ \underbrace{v^{\mathsf T}}_{1 \times 2}\ \underbrace{\operatorname{diag}(\mathbf 1[z > 0])}_{2 \times 2}\ \underbrace{W}_{2 \times 2} = (-5)\,(1, -1)\begin{bmatrix}0 & 0\\ 0 & 1\end{bmatrix}\begin{bmatrix}1 & -1\\ 2 & 1\end{bmatrix} = (-5)(0, -1)\begin{bmatrix}1 & -1\\ 2 & 1\end{bmatrix} = (10, 5).

Multiplying from the left, every intermediate result is a row of two numbers (the work of backpropagation); a finite-difference check of the loss agrees with $(10, 5)$ to nine digits. This product is a gradient of the input, and the transpose form $\nabla h = Dg^{\mathsf T}\nabla f$ of the corollary is what the library computes: a row (or column) times each Jacobian, never a Jacobian times a Jacobian.

The size of that product is also why deep networks are hard to train. With $L$ layers, the gradient is a product of $L$ Jacobians. If each shrinks lengths by a factor $0.9$ , after $L = 50$ layers the signal is $0.9^{50} \approx 0.005$ of its size (vanishing gradients); if each stretches by $1.1$ , it is $1.1^{50} \approx 117$ times larger (exploding gradients). The conditioning of 3.5 controls which happens. Libraries compute either $Df(x)\,v$ (forward mode, one input direction at a time, for example JAX’s jvp) or $u^{\mathsf T}Df(x)$ (reverse mode, one output at a time, vjp). For a loss, with a million inputs and one output, reverse mode gets the whole gradient in one sweep.

Python labCheck the chain rule numerically, then count the cost of the product

import numpy as np

def jac(F, x, h=1e-6):
    x = np.asarray(x, float); cols = []
    for j in range(len(x)):
        e = np.zeros_like(x); e[j] = h
        cols.append((F(x + e) - F(x - e)) / (2 * h))
    return np.array(cols).T

g = lambda p: np.array([p[0] + 2 * p[1]**2 + 3 * p[2]**3, 2 * p[1] - p[0]**2])
f = lambda q: np.array([np.exp(q[0] + 2 * q[1]), np.sin(q[1] + 2 * q[0])])
a = np.array([1.0, -1.0, 1.0])

Dh = jac(lambda p: f(g(p)), a)          # Jacobian of the composite, numerically
prod = jac(f, g(a)) @ jac(g, a)         # Df at g(a), times Dg at a
print(np.round(Dh, 6)); print(np.round(prod, 6))
print("largest difference:", abs(Dh - prod).max(), "  exact (2,3) entry:", 18 * np.cos(9))

# a chain of L square Jacobians, multiplied in two orders
rng = np.random.default_rng(0)
n, L = 300, 10
Js = [rng.normal(size=(n, n)) / np.sqrt(n) for _ in range(L)]
row = rng.normal(size=(1, n))
left = row
for J in Js: left = left @ J                  # row times matrix, L times
right = Js[-1]
for J in reversed(Js[:-1]): right = J @ right # matrix times matrix, L - 1 times
print("same answer:", np.allclose(left, row @ right))
mults_left, mults_right = L * n * n, (L - 1) * n**3 + n * n
print("multiplications:", mults_left, "versus", mults_right, " ratio", round(mults_right / mults_left))

Try this. Run it. The finite-difference Jacobian of the composite and the product of the two Jacobians agree to about nine digits, and the entry in row 2, column 3 matches the exact value 18 cos 9 = -16.40 of the worked example. Change the evaluation point a and the difference stays tiny. Change the step h=1e-6 to 1e-2: the largest difference jumps to about 0.07 (a large step measures the curve, not the tangent). In the second half, the left-to-right and right-to-left products agree, but the count of multiplications differs by a factor of about 270: this is the reason backpropagation multiplies from the output end, when the output is a single number.

Traps

Watch out· Evaluate the outer derivative at g(a), not at a

For $h(x) = f(g(x))$ the matrix $Df$ is taken at the point $g(a)$ . In Apostol 8.22 #14 the point $a = (1, -1, 1)$ goes to $b = (6, -3)$ , and the cosines in $Df$ are $\cos(y + 2x) = \cos9$ , not $\cos(-1 + 2)$ . Compute $g(a)$ first, always.

Watch out· The order of the matrices is not up to you

$Dh = Df\,Dg$ and not $Dg\,Df$ . The shapes tell you: $(m \times n)(n \times p)$ works, and the other order usually does not even have matching sizes. When it does (both square), the answer is still different, because matrix multiplication does not commute (2.5).

Watch out· The partial of f inside a composite is itself a composite

When differentiating $f_x(X(t), Y(t))$ or $f_u(xy, x/y)$ again, apply the chain rule to it. Leaving it as " $f_{xx}$ " loses the term $f_{xy}Y'$ . This was the content of 8.17 #1(b), of the unfinished page of Lecture 23, and of the second half of the Fall 2025 final’s problem 5.

Watch out· Total and partial derivatives: say which variables move

If $f(x, y)$ is composed with $y = Y(x)$ , the derivative of $F(x) = f(x, Y(x))$ is $F' = f_x + f_yY'$ . The symbol $\frac{\partial f}{\partial x}$ means “hold $y$ fixed”, and $\frac{dF}{dx}$ means “let $y$ move with $x$ ”. They are different numbers, and cancelling ” $\partial x$ with $\partial x$ ” in the fraction picture would give $F' = f_y Y'$ or $f_x$ alone. In the buoy problem the forgotten term was $f_t$ , the change of the temperature at a place that the buoy does not even leave.

Watch out· A differentiable bijection need not have a differentiable inverse

$x \mapsto x^3$ is differentiable and one-to-one onto $\mathbb R$ , and its inverse $y^{1/3}$ is not differentiable at $0$ . The derivative-of-an-inverse theorem assumes a differentiable inverse, and the example shows the assumption is doing work. The test that $\det DF \ne 0$ is necessary for a differentiable inverse; sufficient only locally, with continuous partials (the inverse function theorem, previewed above).

Watch out· Mixed partials: equal only under a continuity hypothesis

Every second-derivative formula above collected $f_{xy}$ and $f_{yx}$ into $2f_{xy}$ . That is legitimate when the second partials are continuous. 5.5 proves it and gives the function where it fails.

Problems

checkApostol 8.22 #1

Let $F(t) = e^{\sin t}$ , $g(x, y) = \cos(x^2 + y^2)$ and $f(x, y) = F(g(x, y))$ . Find $f_x$ and $f_y$ by the chain rule.

Solution

The outer function is of one variable, so by the corollary $\nabla f = F'(g)\,\nabla g$ . $F'(t) = e^{\sin t}\cos t$ , and $\nabla g = -\sin(x^2 + y^2)\,(2x, 2y)$ . Hence $f_x = -2x\sin(x^2 + y^2)\,e^{\sin g}\cos g$ and $f_y = -2y\sin(x^2 + y^2)\,e^{\sin g}\cos g$ , where $g = \cos(x^2 + y^2)$ . Check: along $x$ at $y = 0$ , $f = e^{\sin\cos(x^2)}$ and the one-variable chain rule gives the same.

check

$g: \mathbb R^3 \to \mathbb R^2$ and $f: \mathbb R^2 \to \mathbb R^4$ are differentiable. What shape is $D(f \circ g)$ ? Is $g \circ f$ defined?

Solution

$D(f \circ g) = Df\,Dg$ is $(4 \times 2)(2 \times 3) = 4 \times 3$ . The composite $g \circ f$ would feed the four outputs of $f$ into $g$ , which takes three inputs, so it is not defined.

checkApostol 8.22 #2

The substitution $u = (x - y)/2$ , $v = (x + y)/2$ changes $f(u, v)$ into $F(x, y)$ . Express $F_x$ and $F_y$ by $f_u$ and $f_v$ .

Solution

$u_x = \frac12$ , $u_y = -\frac12$ , $v_x = \frac12$ , $v_y = \frac12$ . So $F_x = \frac12(f_u + f_v)$ and $F_y = \frac12(-f_u + f_v)$ . (Then $F_x^2 - F_y^2 = f_uf_v$ .)

checkApostol 8.17 #3(c)

Evaluate the directional derivative of $f(x, y, z) = x^2 + y^2 - z^2$ at $(3, 4, 5)$ along the curve of intersection of the surfaces $2x^2 + 2y^2 - z^2 = 25$ and $x^2 + y^2 = z^2$ .

Solution

The point lies on both surfaces ( $18 + 32 - 25 = 25$ and $9 + 16 = 25$ ). The tangent direction is the cross product of the two normals, $(12, 16, -10)\times(6, 8, -10) = (-80, 60, 0)$ , which is parallel to $(-4, 3, 0)$ . And $\nabla f(3, 4, 5) = (6, 8, -10)$ , so $\nabla f\cdot(-4, 3, 0)/5 = (-24 + 24)/5 = 0$ . In fact $f = x^2 + y^2 - z^2 = 0$ on the second surface, hence on the whole curve (a horizontal circle of radius 5 at height 5), so its derivative along the curve is $0$ .

useApostol 8.22 #4(b) and (c)

The equations $x = st$ , $y = s/t$ turn $f(x, y)$ into $F(s, t)$ . Find $F_s$ , $F_t$ and $F_{ss}$ . Repeat for $x = (s - t)/2$ , $y = (s + t)/2$ , and find $F_{ss} + F_{tt}$ and $F_{st}$ . Assume equal mixed partials.

Solution

First substitution: $x_s = t$ , $y_s = 1/t$ , $x_t = s$ , $y_t = -s/t^2$ . $F_s = t\,f_x + \frac1t\,f_y$ , $F_t = s\,f_x - \frac s{t^2}f_y$ . For $F_{ss}$ , the coefficients $t$ and $1/t$ do not depend on $s$ , so $F_{ss} = t\big(tf_{xx} + \frac1tf_{xy}\big) + \frac1t\big(tf_{xy} + \frac1tf_{yy}\big) = t^2f_{xx} + 2f_{xy} + \frac1{t^2}f_{yy}$ . Second substitution: $F_s = \frac12(f_x + f_y)$ , $F_t = \frac12(-f_x + f_y)$ , $F_{ss} = \frac14(f_{xx} + 2f_{xy} + f_{yy})$ , $F_{tt} = \frac14(f_{xx} - 2f_{xy} + f_{yy})$ , $F_{st} = \frac14(-f_{xx} + f_{yy})$ . So $F_{ss} + F_{tt} = \frac12(f_{xx} + f_{yy})$ , a rotation of the axes by $45^\circ$ and a scaling by $\frac1{\sqrt2}$ in each, which multiplies the Laplacian by $\frac12$ .

useApostol 8.9 #19

Let $z = u(x, y)\,e^{ax + by}$ where $u_{xy} = 0$ . Find constants $a, b$ such that $z_{xy} - z_x - z_y + z = 0$ .

Solution

Product rule with the chain rule on the exponential: $z_x = e^{ax + by}(u_x + au)$ , $z_y = e^{ax + by}(u_y + bu)$ , and $z_{xy} = e^{ax + by}(u_{xy} + bu_x + au_y + abu) = e^{ax + by}(bu_x + au_y + abu)$ . So

z_{xy} - z_x - z_y + z = e^{ax + by}\big[(b - 1)u_x + (a - 1)u_y + (ab - a - b + 1)u\big] = e^{ax + by}\big[(b - 1)u_x + (a - 1)u_y + (a - 1)(b - 1)u\big].

Taking $a = b = 1$ makes every coefficient zero, whatever $u$ is. (For $u$ with $u_x$ , $u_y$ , $u$ independent, such as $u = x + y$ and $u = 1$ , the choice $a = b = 1$ is also forced.)

useApostol 8.22 #5

For $h(r, \theta) = f(r\cos\theta, r\sin\theta)$ find $h_{r\theta}$ in terms of the partials of $f$ . Check it on $f = x^2y$ at $r = 1$ , $\theta = \pi/2$ .

Solution

Differentiate $h_r = \cos\theta\,f_x + \sin\theta\,f_y$ in $\theta$ : $h_{r\theta} = -\sin\theta\,f_x + \cos\theta\,f_y + \cos\theta(-r\sin\theta\,f_{xx} + r\cos\theta\,f_{xy}) + \sin\theta(-r\sin\theta\,f_{xy} + r\cos\theta\,f_{yy})$ , which collects to $-\sin\theta\,f_x + \cos\theta\,f_y + r\big(-\sin\theta\cos\theta\,f_{xx} + (\cos^2\theta - \sin^2\theta)f_{xy} + \sin\theta\cos\theta\,f_{yy}\big)$ . For $f = x^2y$ : $h = r^3\cos^2\theta\sin\theta$ , $h_r = 3r^2\cos^2\theta\sin\theta$ , $h_{r\theta} = 3r^2(\cos^3\theta - 2\cos\theta\sin^2\theta)$ , which is $0$ at $\theta = \pi/2$ . By the formula at $(x, y) = (0, 1)$ : $f_x = 0$ , $f_y = 0$ , $f_{xx} = 2y = 2$ , $f_{xy} = 2x = 0$ , $f_{yy} = 0$ , and the cross coefficient $\sin\theta\cos\theta = 0$ : all terms vanish, giving $0$ .

useApostol 8.17 #5

The equations $e^u\cos v = x$ and $e^u\sin v = y$ define $u$ and $v$ as functions of $(x, y)$ . Find explicit formulas, valid for $x > 0$ , and show that $\nabla u$ and $\nabla v$ are perpendicular at each point.

Solution

Square and add: $e^{2u} = x^2 + y^2$ , so $u = \frac12\ln(x^2 + y^2)$ . Divide: $\tan v = y/x$ , so $v = \arctan(y/x)$ for $x > 0$ . Then $\nabla u = \dfrac{(x, y)}{x^2 + y^2}$ and $\nabla v = \dfrac{(-y, x)}{x^2 + y^2}$ , whose dot product is $\dfrac{-xy + yx}{(x^2 + y^2)^2} = 0$ . Another route, by the theorem of this chapter: the rows of $DG$ are $\nabla u$ and $\nabla v$ , and $DG = (DF)^{-1}$ for $F(u, v) = (e^u\cos v, e^u\sin v)$ , where $DF = e^u\begin{bmatrix}\cos v & -\sin v\\ \sin v & \cos v\end{bmatrix}$ is a scaled rotation; the inverse of a scaled rotation is a scaled rotation, whose rows are perpendicular.

proveApostol 8.22 #12

Let $h(x) = f(g(x))$ with $g = (g_1, \dots, g_n)$ differentiable at $a$ and $f$ a scalar field differentiable at $g(a)$ . Prove $\nabla h(a) = \sum_{k=1}^n D_kf(g(a))\,\nabla g_k(a)$ .

Solution

Plan. Apply the chain rule coordinate by coordinate. For each $j$ the chain rule gives $D_jh(a) = \sum_kD_kf(b)\,D_jg_k(a)$ , where $b = g(a)$ . The vector whose $j$ -th entry is this sum is $\sum_kD_kf(b)\,(D_1g_k(a), \dots, D_ng_k(a)) = \sum_kD_kf(b)\,\nabla g_k(a)$ , since $\nabla g_k(a)$ has $D_jg_k(a)$ as its $j$ -th entry. $\blacksquare$

prove

Use the chain rule to prove the product rule $\nabla(uv) = u\nabla v + v\nabla u$ for differentiable scalar fields $u, v$ .

Solution

Plan. Write $uv = m(u, v)$ with $m(s, t) = st$ .

Let $g(x) = (u(x), v(x))$ , so $uv = m \circ g$ . $\nabla m = (t, s)$ , so $Dm(g(x)) = (v(x), u(x))$ and $Dg$ has rows $\nabla u^{\mathsf T}$ and $\nabla v^{\mathsf T}$ . By the chain rule $D(uv) = (v, u)\begin{bmatrix}\nabla u^{\mathsf T}\\ \nabla v^{\mathsf T}\end{bmatrix} = v\nabla u^{\mathsf T} + u\nabla v^{\mathsf T}$ . Transposing, $\nabla(uv) = v\nabla u + u\nabla v$ . $\blacksquare$

provenew (Euler's identity)

A scalar field $f$ on $\mathbb R^n$ is homogeneous of degree $k$ if $f(tx) = t^kf(x)$ for all $t > 0$ . If $f$ is differentiable, prove $x\cdot\nabla f(x) = k\,f(x)$ .

Solution

Plan. Differentiate $t \mapsto f(tx)$ at $t = 1$ in two ways.

Fix $x$ and let $r(t) = tx$ , so $r'(t) = x$ . By the chain rule for a path, $\frac{d}{dt}f(tx) = \nabla f(tx)\cdot x$ , which at $t = 1$ is $\nabla f(x)\cdot x$ . On the other side, $f(tx) = t^kf(x)$ has derivative $kt^{k-1}f(x)$ , which at $t = 1$ is $k f(x)$ . Equating the two gives $x\cdot\nabla f(x) = kf(x)$ . $\blacksquare$ (Example: $f = x^2y + y^3$ has degree 3 and $x f_x + y f_y = 2x^2y + x^2y + 3y^3 = 3f$ .)

prove

Prove that if $F: U \to V$ is a diffeomorphism, then $\det D(F^{-1})(F(x)) = 1/\det DF(x)$ , and $\det DF(x) \ne 0$ for every $x$ .

Solution

Plan. Take determinants in the identity $D(F^{-1})(F(x))\,DF(x) = I$ .

By the theorem of this chapter, $DG(F(x))\,DF(x) = I$ with $G = F^{-1}$ . Both matrices are $n \times n$ , so the product rule for determinants gives $\det DG(F(x))\cdot\det DF(x) = \det I = 1$ . A product of two numbers equals $1$ , so neither is $0$ , and $\det DG(F(x)) = 1/\det DF(x)$ . $\blacksquare$ For polar coordinates: $\det DP = r$ , so the inverse map $(x, y) \mapsto (r, \theta)$ has Jacobian determinant $1/r = 1/\sqrt{x^2 + y^2}$ , which at $(1, 1)$ is $1/\sqrt2$ ; the matrix $\begin{bmatrix}\frac{\sqrt2}2 & \frac{\sqrt2}2\\ -\frac12 & \frac12\end{bmatrix}$ has determinant $\frac{\sqrt2}4 + \frac{\sqrt2}4 = \frac{\sqrt2}2 = 1/\sqrt2$ . ✓

provenew (uses 5.2)

If $f$ is differentiable on a ball and $f(x)$ depends only on $\lVert x \rVert$ , say $f(x) = \phi(\lVert x \rVert)$ for a differentiable function $\phi$ of one variable, show $\nabla f(x) = \phi'(\lVert x \rVert)\,\dfrac{x}{\lVert x \rVert}$ for $x \ne O$ . What does this say about the level sets?

Solution

Plan. Compose $\phi$ with the field $\lVert x \rVert$ , whose gradient was found in 5.2.

By the fourth row of the table, $\nabla(\phi\circ\lVert\cdot\rVert) = \phi'(\lVert x \rVert)\,\nabla\lVert x \rVert$ , and $\nabla\lVert x \rVert = x/\lVert x \rVert$ (5.2, the cone). So $\nabla f$ points along the radius, and the level sets of $f$ are spheres, to which the gradient is perpendicular. $\blacksquare$ (For $f = \ln\lVert x \rVert$ , $\phi = \ln$ , we get $\nabla f = x/\lVert x \rVert^2$ , matching the $u$ of 8.17 #5.)

stretch

Suppose $F: U \to V$ is a bijection from an open set of $\mathbb R^n$ to an open set of $\mathbb R^m$ which is differentiable with a differentiable inverse. Prove that $n = m$ .

Solution

Plan. The chain rule gives two matrix identities whose sizes force $n = m$ .

Write $G = F^{-1}$ . Then $DG\,DF = I_n$ ( $G \circ F$ is the identity on $U$ ) and $DF\,DG = I_m$ ( $F \circ G$ is the identity on $V$ ), where $DF$ is $m \times n$ and $DG$ is $n \times m$ . The first identity says $DF$ has rank $n$ (it has a left inverse, so it is one-to-one), which needs $n \le m$ . The second says $DG$ has rank $m$ , which needs $m \le n$ . Therefore $n = m$ . $\blacksquare$ This is a baby case of the theorem that open subsets of $\mathbb R^n$ and $\mathbb R^m$ are not diffeomorphic for $n \ne m$ ; the statement for homeomorphisms is much harder (invariance of domain, MATH 551).

stretchd'Alembert, via the second-derivative chain rule

Let $u(x, t)$ have continuous second partials on $\mathbb R^2$ and satisfy $u_{tt} = c^2u_{xx}$ . Prove $u(x, t) = F(x - ct) + G(x + ct)$ for some twice differentiable $F, G$ .

Solution

Plan. Change variables to $\xi = x - ct$ , $\eta = x + ct$ , where the equation becomes $U_{\xi\eta} = 0$ , then integrate twice.

Put $U(\xi, \eta) = u\big(\frac{\xi + \eta}2, \frac{\eta - \xi}{2c}\big)$ , so $u(x, t) = U(x - ct, x + ct)$ . By the second-derivative chain rule (Application above), $u_{tt} - c^2u_{xx} = -4c^2U_{\xi\eta}$ , so $U_{\xi\eta} = 0$ on $\mathbb R^2$ . Then $U_\eta$ has $\partial_\xi(U_\eta) = 0$ on the whole plane, so for each fixed $\eta$ , $\xi \mapsto U_\eta(\xi, \eta)$ has derivative $0$ on all of $\mathbb R$ and is constant (1.4): $U_\eta(\xi, \eta) = \psi(\eta)$ for a function $\psi$ of $\eta$ alone. Let $G$ be an antiderivative of $\psi$ . Then $\partial_\eta\big(U - G(\eta)\big) = 0$ , so for each $\xi$ the function $\eta \mapsto U(\xi, \eta) - G(\eta)$ is constant: $U(\xi, \eta) = F(\xi) + G(\eta)$ with $F(\xi) = U(\xi, 0) - G(0)$ . $\blacksquare$

Summary

Chain rule. If $g$ is differentiable at $a$ and $f$ at $b = g(a)$ , then $f \circ g$ is differentiable at $a$ and $D(f \circ g)(a) = Df(g(a))\,Dg(a)$ : the matrices multiply, the outer one evaluated at the image point and written on the left. The proof substitutes one first-order formula into the other; the leftovers are small because linear maps are continuous and $\lVert g(a + y) - g(a) \rVert \le M\lVert y \rVert$ .
Every formula in the lectures is a shape of this product: a path into a scalar field ( $\nabla f\cdot r'$ ), a change of variables ( $h_s = f_x\,x_s + f_y\,y_s$ ), polar coordinates ( $h_\theta = -r\sin\theta\,f_x + r\cos\theta\,f_y$ ), the gradient of a composite ( $\nabla h = Dg^{\mathsf T}\nabla f$ ).
Inverse. If $F$ has a differentiable inverse $G$ then $DG(F(x)) = DF(x)^{-1}$ and $\det DF \ne 0$ (Practice Final 10). A differentiable bijection like $x^3$ need not have a differentiable inverse.
The Jacobian determinant is the local volume scaling; the chain rule makes it multiply along a composition, and invert along an inverse.
Second derivatives. Differentiate, then apply the chain rule again to each partial of $f$ , with the product rule throughout. Along a path, $g'' = r'^{\mathsf T}H\,r' + \nabla f\cdot r''$ ; for a change of variables, $H_h = Dg^{\mathsf T}H_fDg + \sum_kf_kH_{g_k}$ . Mixed partials were taken equal; 5.5 says when that is legal.
Applications: polar Laplacian $f_{xx} + f_{yy} = h_{rr} + h_r/r + h_{\theta\theta}/r^2$ , d’Alembert’s solution of the wave equation, and backpropagation as a product of Jacobians multiplied from the output end ( $0.9^{50} \approx 0.005$ , $1.1^{50} \approx 117$ ).

Watch and read

The same ideas explained by someone else. Use these when an explanation here does not land.

The picture: circles into ellipses into ellipses

The idea, in words

The chain rule

The chain rule for fields

Special cases

The gradient of a composite

Changing variables

The inverse of a field has the inverse matrix as its derivative

Jacobian determinant

Jacobian determinants multiply

Diffeomorphism

The derivative of an inverse field

Second derivatives of a composite

Second derivative along a path

Second derivatives of a composite field

Where it is used

Traps

Problems

Summary

Watch and read

Where this sits

Comes from

Leads to

Same idea elsewhere