5.3 The total derivative, the Jacobian matrix and tangent planes

A field is differentiable at a point when, near it, the field equals a constant plus a linear map plus an error that is small compared with the step. That linear map is the total derivative; its matrix is the Jacobian, which for a scalar field is the gradient. It implies continuity, it contains every directional derivative, and it gives tangent planes and linear approximations.

In one variable (1.4) a function is differentiable at $a$ when it looks like a line near $a$ : $f(a + h) = f(a) + f'(a)\,h + hE(h)$ with $E(h) \to 0$ . The number $f'(a)$ is the slope, and multiplying by it is a linear map from $\mathbb R$ to $\mathbb R$ (2.3). In many variables the right object is a linear map from $\mathbb R^n$ to $\mathbb R^m$ , and its matrix is the Jacobian. The chapter before showed that having a derivative in each direction is not enough (it did not even give continuity). Demanding a good linear approximation is, and it gives everything we wanted: continuity, a gradient that works in every direction, tangent planes, and the chain rule of 5.4. Chapter 3.8 used the gradient and the Jacobian without definitions; this chapter supplies them.

Before you start

1.4: the derivative as the slope of the best line. 2.5: a linear map is a matrix whose columns are the images of the unit vectors. 5.2: directional derivatives, partial derivatives and the gradient, and the example where they exist without continuity.

Quick checkIn

f(a + h) = f(a) + mh + E(h)

, what does 'differentiable at

a

with derivative

m

' say about

E(h)

Quick checkA linear map

T: \mathbb R^3 \to \mathbb R^2

has a matrix of what shape, and what are its columns?

Quick checkFor

f(x, y) = \dfrac{xy^2}{x^2 + y^4}

(with

f(0, y) = 0

), all directional derivatives exist at the origin. What went wrong in 5.2?

The picture: zoom until it is flat

Drag the dot to a point of the surface and look at the green patch: the tangent plane, the graph of the best linear approximation. Slide the step length down. The vertical purple segment (true value minus plane value) shrinks much faster than the step itself, and the number “error divided by step” in the box under the picture falls towards $0$ . Do this for $\cos(x + y)$ and the saddle too. For a vector field the same idea says that a tiny disc around $a$ is sent by $F$ to something that looks like an ellipse, the image of the disc under a matrix:

The purple curve is the true image of the tiny circle (after subtracting $F(a)$ and dividing by the radius). The dashed green ellipse is what the Jacobian matrix $J$ does to the unit circle. Make the radius $\rho$ small, and they are the same curve. Try the “swirl” field and move the dot around: the matrix changes from point to point, but at each point the circle becomes an ellipse.

The idea, in words

The one-variable equation says: near $a$ , $f$ is a constant $f(a)$ , plus a linear function of the step $h$ , plus an error. In many variables, the step $v = x - a$ is a vector, the linear function is a linear map $T_a(v)$ , and the error must be small compared with the length of the step. You cannot divide by a vector, so the one-variable condition " $E(h)/h \to 0$ " is rewritten as “error $= \lVert v \rVert\,E(a, v)$ with $E \to 0$ ”. This is Apostol’s 8.11, Spagnolie’s Lecture 21, and Young’s and Fréchet’s idea.

Why· Why a linear map, and why 'all directions at once'

If you walk from $a$ in the direction $y$ , the error condition says $f(a + hy) - f(a) = T_a(hy) + o(h) = h\,T_a(y) + o(h)$ . So $T_a(y)$ must be the directional derivative $f'(a; y)$ . The total derivative is the map that remembers all the directional derivatives at once and insists that they depend linearly on $y$ and that the approximation is good uniformly in the direction. The sneaky function of 5.2 failed exactly these two demands: its directional derivative $q^2/p$ is not linear in $y$ , and its approximation is good along each line but not along the parabola.

Where this came from· Jacobi's functional determinants, and Young and Fréchet

Carl Jacobi’s 1841 paper “De determinantibus functionalibus” in Crelle’s journal studied the determinant of the square array of partial derivatives of $n$ functions of $n$ variables: the determinant that now carries his name. He showed that $n$ functions of $n$ variables are mutually dependent exactly when this determinant vanishes identically. The same determinant is the number that says how a small region is stretched, the area scaling in the picture above. The idea of a derivative of a function of several variables as a linear map approximating it, uniformly in the direction, came later: Apostol’s text credits W. H. Young (1908) and Maurice Fréchet (1911), the latter “in a more general context”, meaning that Fréchet’s definition works on any normed space. That generality is why 521 states it that way.

The definition

Definition T

Differentiable field

Let $F: S \to \mathbb R^m$ with $S \subseteq \mathbb R^n$ , and let $a$ be an interior point of $S$ , so a ball $B(a, r)$ lies in $S$ . We say $F$ is differentiable at $a$ if there is a linear map $T_a: \mathbb R^n \to \mathbb R^m$ and a function $E(a, v)$ (with values in $\mathbb R^m$ , defined for $\lVert v \rVert < r$ , with $E(a, 0) = 0$ ) such that

F(a + v) = F(a) + T_a(v) + \lVert v \rVert\,E(a, v) \quad (\lVert v \rVert < r), \qquad E(a, v) \to 0 \text{ as } v \to 0.

An equivalent form, with no $E$ : $\displaystyle\lim_{v \to 0}\frac{\lVert F(a + v) - F(a) - T_a(v) \rVert}{\lVert v \rVert} = 0$ . (Define $E(a, v)$ as the quotient inside, for $v \ne 0$ .) For $m = 1$ this is Apostol’s definition of a differentiable scalar field.

In one line $F$ is differentiable at the interior point $a$ if $F(a + v) = F(a) + T_a(v) + \lVert v \rVert E(a, v)$ for small $v$ , where $T_a$ is a linear map and $E(a, v) \to 0$ as $v \to 0$ .

Why it existsIt says that near $a$ the field is a constant plus a linear map plus an error that is small compared with the step, which is the one-variable zoom picture in $n$ dimensions and the definition that supports the chain rule.

Definition T

Total derivative

When $F$ is differentiable at $a$ , the linear map $T_a$ is called the total derivative of $F$ at $a$ , written $F'(a)$ (Apostol) or $DF(a)$ (Spagnolie), so $F'(a)(v) = T_a(v)$ . Note (Apostol’s warning): the total derivative is a linear map, not a number; the value $T_a(v)$ is a vector for each $v$ . It is unique, by the theorem below. In one variable ( $n = m = 1$ ) every linear map is multiplication by a number, $T_a(h) = f'(a)h$ , and the definition is the form of 1.4.

In one lineThe linear map $T_a$ in the definition of a differentiable field; it is unique, and it is written $F'(a)$ or $DF(a)$ .

Why it existsIt is the object that plays the role of the number $f'(a)$ : the best linear approximation of $F$ at $a$ , with the error beyond first order.

Worked example new (this is the product rule)

Show from the definition that

f(x, y) = xy

is differentiable at every

a = (a_1, a_2)

with total derivative

T_a(v) = a_2v_1 + a_1v_2

Solution.

Expand $f(a + v)$ . $f(a + v) = (a_1 + v_1)(a_2 + v_2) = a_1a_2 + a_2v_1 + a_1v_2 + v_1v_2$ . Multiply out; $a$ is fixed, $v$ is the small step.
Read off the pieces. $f(a) = a_1a_2$ ; $T_a(v) = a_2v_1 + a_1v_2$ is linear in $v$ (a row times a column); the leftover is $v_1v_2$ . Constant plus linear plus error.
Bound the error. $|v_1v_2| \le \frac12(v_1^2 + v_2^2) = \frac12\lVert v \rVert^2$ . $(v_1 - v_2)^2 \ge 0$ , that is, $2v_1v_2 \le v_1^2 + v_2^2$ , and the same with $v_2 \to -v_2$ .
Divide by the length. $\dfrac{|v_1v_2|}{\lVert v \rVert} \le \dfrac12\lVert v \rVert \to 0$ . So $E(a, v) = \dfrac{v_1v_2}{\lVert v \rVert}$ tends to $0$ as $v \to 0$ .

Answer. $xy$ is differentiable everywhere, with $T_a(v) = (a_2, a_1)\cdot v$ . The error $v_1v_2$ is of second order, and it is the cross term that the product rule of 1.4 treated with the add-and-subtract trick.

Worked example new (compare 3.8, Problems)

Find the total derivative of a linear field

F(x) = Ax + b

, where

A

m \times n

Solution.

Expand. $F(a + v) = A(a + v) + b = (Aa + b) + Av = F(a) + Av$ . $A$ is linear.
Read off. The error term is $0$ exactly, so $E \equiv 0$ , and $T_a(v) = Av$ for every $a$ . There is nothing nonlinear left to estimate.

Answer. $F'(a) = A$ at every point: a linear map is its own best linear approximation, and its derivative does not depend on where you are.

What the total derivative contains

Theorem T Apostol Thm 8.5 and 8.12; Spagnolie Lecture 21

Directional derivative as a dot product with the gradient

Suppose $f: S \to \mathbb R$ is differentiable at $a$ with total derivative $T_a$ . Then for every $y \in \mathbb R^n$ the derivative $f'(a; y)$ exists and

T_a(y) = f'(a; y) = \sum_{k=1}^n y_kD_kf(a) = \nabla f(a)\cdot y.

In particular all the partial derivatives exist, $T_a$ is the only linear map that works, and it is the linear function $y \mapsto \nabla f(a)\cdot y$ .

In one lineIf a scalar field $f$ is differentiable at $a$ , then $f'(a; y)$ exists for every $y$ and equals $T_a(y) = \nabla f(a)\cdot y$ ; in particular $T_a$ is unique.

Why it is trueIt proves the formula borrowed in 5.2, shows the total derivative is determined by the field, and shows it is the gradient read as a linear function.

ProofApostol 8.5; Spagnolie Lecture 21

The idea: Take the step v = hy along a line. By linearity, T_a(hy) = h T_a(y), so dividing by h leaves T_a(y) plus a term that goes to 0. The sum formula then follows from writing y in the unit vectors.

Let

y \ne O

and

h \ne 0

small. Put

v = hy

in the definition.

The case

y = O

: both sides are

0

f(a + hy) - f(a) = h\,T_a(y) + |h|\,\lVert y \rVert\,E(a, hy)

Definition of a differentiable field with

v = hy

; linearity gives

T_a(hy) = hT_a(y)

;

\lVert hy \rVert = |h|\,\lVert y \rVert

\dfrac{f(a + hy) - f(a)}{h} = T_a(y) \pm \lVert y \rVert\,E(a, hy)

Divide by

h

. Here

|h|/h = \pm1

h \to 0

the right side tends to

T_a(y)

. So

f'(a; y)

exists and equals

T_a(y)

hy \to O

h \to 0

, so

E(a, hy) \to 0

. The sign

\pm

and the factor

\lVert y \rVert

are bounded.This is where 'small compared with ‖v‖' is used: the error survives division by h and still vanishes.

T_a(y) = \sum_k y_kT_a(e_k) = \sum_k y_kf'(a; e_k) = \sum_k y_kD_kf(a) = \nabla f(a)\cdot y

Write

y = \sum_k y_ke_k

and use that

T_a

is linear.

So any two total derivatives agree on every

y

T_a

is unique.

\blacksquare

T_a(y)

equals a quantity,

f'(a; y)

, that does not depend on

T_a

Proof workshopGive the reason for each line: the total derivative contains all directional derivativesApostol 8.5

Each line of the proof is shown in order. Choose the reason that justifies it, then press Check.

$f(a + hy) - f(a) = T_a(hy) + \lVert hy \rVert\,E(a, hy)$ .
$T_a(hy) = h\,T_a(y)$ and $\lVert hy \rVert = |h|\,\lVert y \rVert$ .
$\dfrac{f(a + hy) - f(a)}{h} = T_a(y) \pm \lVert y \rVert\,E(a, hy)$ .
As $h \to 0$ , $E(a, hy) \to 0$ , so the quotient tends to $T_a(y)$ .
Hence $f'(a; y) = T_a(y) = \sum_k y_k D_k f(a) = \nabla f(a) \cdot y$ .

Theorem T Apostol 8.18; Spagnolie Lecture 23

A vector field is differentiable exactly when its components are

Let $F: S \to \mathbb R^m$ . Then $F$ is differentiable at $a$ if and only if each component $F_i$ is. In that case $F'(a)(y) = F'(a; y) = \big(F_1'(a; y), \dots, F_m'(a; y)\big)$ (for a vector field, $F'(a; y)$ means $\lim_{h \to 0}\frac{F(a + hy) - F(a)}{h}$ , which exists exactly when each component’s limit does) and, in the standard bases, the matrix of $F'(a)$ is the $m \times n$ matrix whose $i$ -th row is $\nabla F_i(a)$ .

In one line $F = (F_1, \dots, F_m)$ is differentiable at $a$ if and only if every $F_i$ is, and then $F'(a)(y) = (\nabla F_1(a)\cdot y, \dots, \nabla F_m(a)\cdot y)$ , whose matrix is the Jacobian.

Why it is trueIt reduces the vector case to the scalar case already done, and tells you how to compute $F'(a)$ : take the gradient of each component and stack them as rows.

ProofApostol 8.18

The idea: A vector is small exactly when all its components are small (the norm comparison of 5.1), so the error E tends to 0 if and only if each component of E does; and a map into R^m is linear exactly when each component is.

F_i(a + v) = F_i(a) + T_{a,i}(v) + \lVert v \rVert E_i(a, v)

, where

T_{a,i}

is the

i

-th component of

T_a

and

E_i

that of

E

Take components of the defining equation: the

i

-th component of

\lVert v \rVert E(a, v)

\lVert v \rVert E_i(a, v)

F

is differentiable, each

T_{a,i}

is linear, and

E \to 0

forces

E_i \to 0

since

|E_i| \le \lVert E \rVert

. So each

F_i

is differentiable.

T_a

is linear exactly when each component

T_{a,i}

is a linear function.

Conversely, if each

F_i

is differentiable with total derivative

T_{a,i}

, put

T_a = (T_{a,1}, \dots, T_{a,m})

, linear, and

E = (E_1, \dots, E_m) \to O

\lVert E \rVert \le \sum_i |E_i|

, as in the proof of the component theorem of 5.1.

T_a(y) = (\nabla F_i(a)\cdot y)_i

, the matrix-times-vector with rows

\nabla F_i(a)

\blacksquare

Theorem above for each scalar field

F_i

T_{a,i}(y) = \nabla F_i(a)\cdot y

, which is row

i

of a matrix times

y

Definition T

Jacobian matrix

For $F: S \to \mathbb R^m$ with all partial derivatives of all components existing at $a$ , the Jacobian matrix of $F$ at $a$ is

J_F(a) = DF(a) = \begin{bmatrix} D_1F_1(a) & D_2F_1(a) & \cdots & D_nF_1(a)\\ \vdots & \vdots & & \vdots\\ D_1F_m(a) & D_2F_m(a) & \cdots & D_nF_m(a)\end{bmatrix} = \frac{\partial(F_1, \dots, F_m)}{\partial(x_1, \dots, x_n)}(a).

Row $i$ is $\nabla F_i(a)$ . Column $j$ is $F'(a; e_j)$ , the image of the $j$ -th unit vector, as in 2.5. When $F$ is differentiable, $J_F(a)$ is the matrix of $F'(a)$ , so $F'(a)(y) = J_F(a)\,y$ . For a scalar field ( $m = 1$ ) it is the single row $\nabla f(a)^{\mathsf T}$ : the gradient is a column and the Jacobian is a row, the distinction flagged in 3.8. When $m = n$ its determinant (2.7) is the Jacobian determinant, the factor by which $F$ scales small areas or volumes.

In one lineThe $m \times n$ matrix $J_F(a)$ of partial derivatives whose entry in row $i$ and column $j$ is $D_jF_i(a)$ : the matrix of the total derivative.

Why it existsIt turns the abstract linear map $F'(a)$ into a table of numbers you can compute and multiply, which is what the chain rule and backpropagation do.

Worked example Lecture 23

Find the Jacobian matrix of

F(x, y, z) = (xy\sin z,\ x^2)

and evaluate

F'(a; y)

a = (2, 0, \pi/2)

Solution.

Components and gradients. $F_1 = xy\sin z$ has $\nabla F_1 = (y\sin z,\ x\sin z,\ xy\cos z)$ . $F_2 = x^2$ has $\nabla F_2 = (2x, 0, 0)$ . Each row of the Jacobian is the gradient of one component.
Stack as rows. $DF = \begin{bmatrix} y\sin z & x\sin z & xy\cos z\\ 2x & 0 & 0\end{bmatrix}$ , a $2 \times 3$ matrix. Two outputs, three inputs: $m \times n = 2 \times 3$ .
Plug in $a$ . At $(2, 0, \frac\pi2)$ : $\sin z = 1$ , $\cos z = 0$ , so $DF(a) = \begin{bmatrix}0 & 2 & 0\\ 4 & 0 & 0\end{bmatrix}$ . The derivative at a fixed point is a matrix of numbers, with no $x, y, z$ left in it.
Apply to a direction. $F'(a; y) = DF(a)\,y = (2y_2,\ 4y_1)$ . Matrix times vector.

Answer. $DF(a) = \begin{bmatrix}0 & 2 & 0\\ 4 & 0 & 0\end{bmatrix}$ and $F'(a; y) = (2y_2, 4y_1)$ . (This $F$ has continuous partials, so by the theorem later in this chapter it is differentiable.)

Your answerFor

F(x, y) = (x^2y,\ x + y)

, what is the entry in row 1, column 1 of the Jacobian matrix at

(1, 2)

Worked example Practice Final 2013, problem 4

Compute the Jacobian matrix of

f(x, y, z) = \big(\log(x + y),\ \cos(2xy),\ e^{x^2y}\big)

Solution.

Row 1. $\nabla\log(x + y) = \Big(\dfrac1{x + y},\ \dfrac1{x + y},\ 0\Big)$ . $z$ does not appear.
Row 2. $\nabla\cos(2xy) = \big(-2y\sin(2xy),\ -2x\sin(2xy),\ 0\big)$ . Chain rule: derivative of $\cos u$ is $-\sin u$ times $\frac{\partial u}{\partial x} = 2y$ .
Row 3. $\nabla e^{x^2y} = \big(2xy\,e^{x^2y},\ x^2e^{x^2y},\ 0\big)$ . Chain rule: $e^u$ times $\frac{\partial u}{\partial x} = 2xy$ and $\frac{\partial u}{\partial y} = x^2$ .

Answer. $Df = \begin{bmatrix}\frac1{x+y} & \frac1{x+y} & 0\\ -2y\sin 2xy & -2x\sin 2xy & 0\\ 2xye^{x^2y} & x^2e^{x^2y} & 0\end{bmatrix}$ . The zero column says that $f$ does not change at all in the $z$ direction.

Differentiable implies continuous

Theorem T Apostol Thm 8.6 and 8.19; Spagnolie Lecture 21

Differentiable fields are continuous

If $F: S \to \mathbb R^m$ is differentiable at the interior point $a$ , then $F$ is continuous at $a$ .

In one lineIf $F$ is differentiable at $a$ , then $F$ is continuous at $a$ .

Why it is trueIt is the property directional derivatives lacked in 5.2; with it the idea of 'derivative' behaves as in one variable.

ProofApostol 8.6, 8.19

The idea: The change F(a+v) − F(a) equals T_a(v) plus an error. Both are bounded by a constant times ‖v‖ (Cauchy–Schwarz for the linear part), so both vanish as v goes to 0.

F(a + v) - F(a) = T_a(v) + \lVert v \rVert E(a, v)

Definition of a differentiable field.

\lVert F(a + v) - F(a) \rVert \le K\lVert v \rVert + \lVert v \rVert\,\lVert E(a, v) \rVert

By the proof of part (c) of 'Building continuous fields' in 5.1, a linear map

T

satisfies

\lVert T(v) \rVert \le K\lVert v \rVert

for a constant

K

. For a scalar field,

K = \lVert\nabla f(a)\rVert

by Cauchy–Schwarz:

|T_a(v)| = |\nabla f(a)\cdot v| \le \lVert\nabla f(a)\rVert\,\lVert v \rVert

The right side is

\lVert v \rVert\,(K + \lVert E \rVert)

, which tends to

0

\lVert v \rVert \to 0

K

is a fixed number and

\lVert E(a, v) \rVert \to 0

F(a + v) \to F(a)

F

is continuous at

a

\blacksquare

Definition of continuity at

a

Quick checkWhich of these is a correct implication for a scalar field at an interior point

a

When does the total derivative exist?

Checking the definition is rarely necessary, because of the following test, which Apostol proves in 8.13 and the lecture left to the book.

Definition A

Continuously differentiable field

A scalar field $f$ is continuously differentiable at $a$ if the partial derivatives $D_1f, \dots, D_nf$ exist in some ball $B(a)$ and are continuous at $a$ . A vector field is if each component is. It is continuously differentiable on a set if it is at every point of it (the class often called $C^1$ ). Every field built from polynomials, $\exp$ , $\sin$ , $\log$ and so on by sums, products, quotients and composition is continuously differentiable where it is defined, because its partials are again such expressions.

In one lineA scalar field is continuously differentiable at $a$ if its partial derivatives exist in some ball around $a$ and are continuous at $a$ .

Why it existsIt is the condition that is easy to check by looking at the formula, and it is strong enough to give differentiability.

Theorem A Apostol Thm 8.7 (8.13)

Continuous partial derivatives imply differentiable

Suppose the partial derivatives $D_1f, \dots, D_nf$ exist in a ball $B(a)$ and are continuous at $a$ . Then $f$ is differentiable at $a$ and $f(a + v) = f(a) + \nabla f(a)\cdot v + \lVert v \rVert E(a, v)$ with $E(a, v) \to 0$ as $v \to 0$ .

In one lineIf $f$ is continuously differentiable at $a$ , then $f$ is differentiable at $a$ , with $T_a(v) = \nabla f(a)\cdot v$ .

Why it is trueIt is the practical test: look at the partial derivatives, and if they are continuous, you have a total derivative and every consequence of it.

ProofApostol 8.13

The idea: Move from a to a + v one coordinate at a time. Each move changes only one variable, so the one-variable mean value theorem applies, and produces a partial derivative at a point near a. Continuity of the partials at a makes those partial derivatives close to their values at a.

Define the points

p_0 = a

p_k = a + \lambda(u_1e_1 + \dots + u_ke_k)

, so

p_n = a + v

Let

\lambda = \lVert v \rVert > 0

and

u = v/\lambda

, a unit vector; take

v

so small that

B(a, \lambda)

lies in the ball

B(a)

f(a + v) - f(a) = \sum_{k=1}^n\big[f(p_k) - f(p_{k-1})\big]

Telescoping sum: the intermediate values cancel in pairs.

f(p_k) - f(p_{k-1}) = \lambda u_k\,D_kf(c_k)

for some

c_k

on the segment from

p_{k-1}

p_k

p_k

and

p_{k-1}

differ only in coordinate

k

, by

\lambda u_k

. The function

t \mapsto f(p_{k-1} + te_k)

is differentiable for

t

between

0

and

\lambda u_k

, with derivative

D_kf

there (the partials exist in the ball); so the one-variable mean value theorem of 1.4 applies.If

u_k = 0

the difference is

0

and any point works.

f(a + v) - f(a) - \nabla f(a)\cdot v = \lambda\sum_k u_k\big[D_kf(c_k) - D_kf(a)\big] = \lVert v \rVert E(a, v)

, where

E(a, v) = \sum_k u_k\big[D_kf(c_k) - D_kf(a)\big]

\nabla f(a)\cdot v = \lambda\sum_k u_kD_kf(a)

Hence

|E(a, v)| \le \sum_k|D_kf(c_k) - D_kf(a)| \to 0

v \to 0

\blacksquare

Every

c_k

lies within

\lambda = \lVert v \rVert

a

(all the points of the path are), so

c_k \to a

v \to 0

; each

D_kf

is continuous at

a

; and

|u_k| \le 1

Proof workshopRebuild the proof that continuous partials give differentiabilityApostol 8.13

Tap the lines in the order a proof would write them. Each line can only use what is already above it.

Watch out· The sufficient condition is not necessary

The converse is false: differentiable does not require continuous partials (example below), and Spagnolie’s Lecture 21 version, ” $f$ is differentiable if $f'$ is continuous”, is a sufficient condition, not an equivalence (the instructor’s own slip, flagged in the digest). Continuous partials imply differentiable implies continuous, and none of the arrows reverses.

The implication table of homework 12

Worked example MATH 375 HW 12 (Apostol 8.14 #11); the printed keys have errors

For a scalar field

f

at an interior point

a

, consider (a)

f

continuous at

a

; (b)

f

differentiable at

a

; (c)

f'(a; y)

exists for every

y

; (d) all first-order partials exist in a neighbourhood of

a

and are continuous at

a

; (e)

\nabla f(a) = O

; (f)

f(x) = \lVert x - a \rVert

for all

x

. Fill in the table of which statement always implies which.

Solution.

Collect the implications that are theorems. (b) implies (a) and (c): differentiable implies continuous, and implies all directional derivatives exist. (d) implies (b): continuous partials imply differentiable; so (d) implies (a), (b), (c). Theorems of this chapter, plus transitivity.
Find a witness field for each kind of failure. Work at $a = O$ in $\mathbb R^2$ .
- $F_1(x, y) = x$ : differentiable with continuous partials, and $\nabla F_1 = (1, 0) \ne O$ . It satisfies (a), (b), (c), (d), and fails (e) and (f).
- $F_2 = \sqrt{|xy|}$ : continuous, both partials are $0$ at $O$ (so (e) holds), but $F_2'(O; (1, 1)) = \lim \frac{|h|}{h}$ does not exist. So (c) fails, hence (b) and (d) fail.
- $F_3 = \frac{xy}{x^2 + y^2}$ with $F_3(O) = 0$ : both partials are $0$ (so (e) holds), but $F_3$ is not continuous at $O$ (it is $\frac12$ on $y = x$ ). So (a) fails, and with it (b), (c), (d).
- $F_4 = \frac{xy^2}{x^2 + y^4}$ with $F_4(0, y) = 0$ (from 5.2): all directional derivatives exist (so (c) holds), the gradient is $O$ (so (e) holds), but it is not continuous, so (a), (b), (d) fail.
- $F_5(x) = \lVert x \rVert$ (this is (f)): continuous, but $F_5'(O; y) = \lim \frac{|h|\,\lVert y \rVert}{h}$ does not exist for $y \ne O$ . So (c), (b), (d) fail, and the partials do not exist, so (e) fails.
- $F_6 = (x^2 + y^2)\sin\frac1{x^2 + y^2}$ with $F_6(O) = 0$ : differentiable at $O$ with $T = 0$ (shown below), so (a), (b), (c), (e) hold, but its partial derivatives are not continuous at $O$ , so (d) fails.
Each non-implication needs one example; six fields give all of them.

Read the table off. Row = hypothesis, column = conclusion; a tick means “always implies”.

	(a)	(b)	(c)	(d)	(e)	(f)
(a) continuous	✓
(b) differentiable	✓	✓	✓
(c) all $f'(a; y)$ exist			✓
(d) continuous partials	✓	✓	✓	✓
(e) $\nabla f(a) = O$					✓
(f) $f = \lVert x - a \rVert$	✓					✓

Why the blanks are blank: (a) does not imply (b), (c), (d) by

F_2

F_5

, nor (e), (f) by

F_1

. (b) does not imply (d) by

F_6

, nor (e), (f) by

F_1

. (c) does not imply (a) by

F_4

(so not (b), (d)), nor (e), (f) by

F_1

. (d) does not imply (e), (f) by

F_1

. (e) implies nothing else, by

F_3

. (f) does not imply (b), (c), (d), (e) by

F_5

Answer. The table above. The differentiability of $F_6$ : $|F_6(v)| \le \lVert v \rVert^2$ , so $F_6(v) = 0 + 0\cdot v + \lVert v \rVert E$ with $E = \lVert v \rVert\sin\frac1{\lVert v \rVert^2} \to 0$ ; yet $\frac{\partial F_6}{\partial x} = 2x\sin\frac1{x^2 + y^2} - \frac{2x}{x^2 + y^2}\cos\frac1{x^2 + y^2}$ , which on the $x$ -axis at $x_n = 1/\sqrt{2\pi n}$ equals $-\frac{2}{x_n} \to -\infty$ . The definition matters for (f). Apostol’s definition of $f'(a; y)$ (8.6) takes $h \to 0$ from both sides, and that is the definition of 5.2. With it, $\frac{f(a + hy) - f(a)}{h} = \frac{|h|\,\lVert y \rVert}{h}$ is $+\lVert y \rVert$ for $h > 0$ and $-\lVert y \rVert$ for $h < 0$ , so the limit does not exist. A one-sided derivative would exist and equal $\lVert y \rVert$ , and (f) would then imply (c). The printed keys differ from this table: both versions mark (b) implies (d), which is false (witness $F_6$ ); both mark row (e) with ticks beyond (e) itself, false (witness $F_3$ ); and both tick (c) in row (f), which holds only for a one-sided derivative, not for the two-sided limit $f'(a; y)$ used here (witness $F_5$ ). Apostol’s printed answer, “(b) implies (a) and (c); (d) implies (a), (b), (c); (f) implies (a)”, agrees with the table.

Quick checkWhich field is differentiable at the origin but has partial derivatives that are not continuous there?

The linearization and its error

Definition T

Linearization

If $F$ is differentiable at $a$ , its linearization (linear approximation) at $a$ is $L(x) = F(a) + F'(a)(x - a)$ . For a scalar field, $L(x) = f(a) + \nabla f(a)\cdot(x - a)$ . The definition says $F(x) - L(x) = \lVert x - a \rVert\,E(a, x - a)$ , which is small compared with $\lVert x - a \rVert$ .

In one lineThe affine function $L(x) = F(a) + F'(a)(x - a)$ that approximates a differentiable field $F$ near $a$ .

Why it existsIt replaces a complicated field by a constant plus a matrix times the displacement, which is easy to compute with, and it is exactly the function whose graph is the tangent plane.

Worked example Lecture 21

Linearize

F(x, y, z) = x^2y\,e^{z^2}

about

a = (1, 2, 0)

and test it at

(1.1, 1.9, 0.1)

Solution.

Value and partials at $a$ . $F(a) = 1^2\cdot 2\cdot e^0 = 2$ . $F_x = 2xy\,e^{z^2} = 4$ , $F_y = x^2e^{z^2} = 1$ , $F_z = x^2y\,e^{z^2}\cdot 2z = 0$ . The lecture’s slip: the printed evaluation point $(1, 1, 0)$ does not give these values; $a = (1, 2, 0)$ does.
Form the linearization. $L(x) = 2 + 4(x - 1) + 1\cdot(y - 2) + 0\cdot z$ . $F(a) + \nabla F(a)\cdot(x - a)$ .
Test at $(1.1, 1.9, 0.1)$ . $L = 2 + 0.4 - 0.1 = 2.3$ ; the true value is $1.21\times1.9\times e^{0.01} = 2.3221$ . Error $0.0221$ .
Error relative to the step. $\lVert v \rVert = \sqrt{0.01 + 0.01 + 0.01} = 0.1732$ , so error $/\lVert v \rVert = 0.128$ . At one tenth of the step, $v = (0.01, -0.01, 0.01)$ , the point is $(1.01, 1.99, 0.01)$ : the true value is $2.03020$ , the plane says $2.03$ , the error is $2.0\times10^{-4}$ and $\lVert v \rVert = 0.01732$ , so the ratio is $0.0117$ . Shrinking $v$ by a factor of $10$ shrinks the error by about $100$ and the ratio by about $10$ : the ratio is proportional to $\lVert v \rVert$ for a smooth field, which is the second-order behaviour 5.6 explains.

Answer. $L(x) = 2 + 4(x - 1) + (y - 2)$ ; $L(1.1, 1.9, 0.1) = 2.3$ against the true $2.3221$ .

Your answerUse the linearization

L(x) = 2 + 4(x - 1) + (y - 2)

of the example to estimate

F(0.9, 2.1, 0.05)

Worked example MATH 375 Final Review, 8.18

Linearize the vector field

F(x, y, z) = (xyz,\ x + y + z)

about

a = (1, 2, 2)

Solution.

Value. $F(a) = (1\cdot2\cdot2,\ 1 + 2 + 2) = (4, 5)$ .
Jacobian. $DF = \begin{bmatrix}yz & xz & xy\\ 1 & 1 & 1\end{bmatrix}$ , so $DF(a) = \begin{bmatrix}4 & 2 & 2\\ 1 & 1 & 1\end{bmatrix}$ . Gradients of the two components as rows.
Linearization. $F(x) \approx F(a) + DF(a)(x - a) = \big(4 + 4(x - 1) + 2(y - 2) + 2(z - 2),\ 5 + (x - 1) + (y - 2) + (z - 2)\big)$ . Matrix times the displacement vector $(x - 1, y - 2, z - 2)$ .
Check numerically. At $(1.1, 2.1, 1.9)$ the true value is $(4.389, 5.1)$ and the linearization gives $(4.4, 5.1)$ . Agreement to the first order; the first component is off by $0.011$ because $xyz$ has second-order terms.

Answer. As in step 3. The linearization of a vector field is a pair of tangent planes, one for each component.

Python labWatch the error shrink faster than the step, or fail to

import numpy as np
rng = np.random.default_rng(1)

def smooth(x, y):  return x**2 * y
def cone(x, y):    return np.hypot(x, y)
def sneaky(x, y):  return np.where(x == 0, 0.0, x * y**2 / (x**2 + y**4 + 1e-300))
def wiggle(x, y):
    s = x**2 + y**2
    return np.where(s == 0, 0.0, s * np.sin(1 / np.where(s == 0, 1, s)))

cases = [("x^2 y at (1,2), gradient (4,1)", smooth, (1.0, 2.0), (4.0, 1.0)),
         ("cone at 0, 'gradient' (0,0)", cone, (0.0, 0.0), (0.0, 0.0)),
         ("sneaky at 0, gradient (0,0)", sneaky, (0.0, 0.0), (0.0, 0.0)),
         ("(x^2+y^2) sin(1/(x^2+y^2)) at 0, gradient (0,0)", wiggle, (0.0, 0.0), (0.0, 0.0))]

th = rng.uniform(0, 2 * np.pi, 4000)
print("worst error/|v| over 4000 random directions and the two parabolas x = +-y^2")
print(f"{'':52s}" + "".join(f"  |v|={r:7.0e}" for r in (1e-1, 1e-2, 1e-3, 1e-4)))
for name, f, a, g in cases:
    row = []
    for r in (1e-1, 1e-2, 1e-3, 1e-4):
        vx, vy = r * np.cos(th), r * np.sin(th)
        # also points on the parabolas x = +-y^2 at distance about r
        vx = np.concatenate([vx, [r**2, -r**2]]); vy = np.concatenate([vy, [r, r]])
        err = np.abs(f(a[0] + vx, a[1] + vy) - f(*a) - (g[0] * vx + g[1] * vy)) / np.hypot(vx, vy)
        row.append(err.max())
    print(f"{name:52s}" + "".join(f"  {e:11.2e}" for e in row))

Try this. Run it. For x²y the ratio error/|v| falls by 10 each time |v| falls by 10: the error is of second order. For the cone and the sneaky function it does not fall: no linear map approximates them. Notice for the cone that the ratio is exactly 1 for every |v|: whatever 'gradient' you guess, the corner defeats it. For (x²+y²) sin(1/(x²+y²)) the ratio falls even though the partials misbehave. Try changing the guessed gradient for the smooth case to (4.1, 1): the ratio then stops falling and levels off near 0.1, because a wrong linear map leaves a first-order error.

The derivative along a curve

Theorem T Apostol Thm 8.8 (8.15); Spagnolie Lecture 22

Rate of change along a curve

Let $r: (\alpha, \beta) \to \mathbb R^n$ be differentiable at $t$ (each coordinate has a derivative, $r'(t)$ is the vector of those derivatives) and let $f$ be a scalar field differentiable at $a = r(t)$ . Then $g = f \circ r$ is differentiable at $t$ and

g'(t) = \nabla f(r(t))\cdot r'(t).

In one lineIf $r$ is differentiable at $t$ and $f$ is differentiable at $r(t)$ , then $g(t) = f(r(t))$ has $g'(t) = \nabla f(r(t))\cdot r'(t)$ .

Why it is trueIt is the chain rule for a path, and the tool that proves the gradient is perpendicular to level sets; it is the special case $m = 1$ , $n_{\text{input}} = 1$ of the chain rule of 5.4.

ProofSpagnolie Lecture 22; Apostol 8.15

The idea: Put v = r(t+h) − r(t) into the first-order formula for f at a, divide by h, and let h go to 0. The linear part gives the dot product with r′; the error part is a bounded ratio times a quantity that goes to 0.

g(t + h) - g(t) = \nabla f(a)\cdot v(h) + \lVert v(h) \rVert\,E(a, v(h))

Differentiability of

f

a = r(t)

, with the step

v(h) = r(t + h) - r(t)

\dfrac{g(t + h) - g(t)}{h} = \nabla f(a)\cdot\dfrac{v(h)}{h} + \dfrac{\lVert v(h) \rVert}{h}\,E(a, v(h))

Divide by

h

The first term tends to

\nabla f(a)\cdot r'(t)

r

is differentiable at

t

, so

\frac{v(h)}{h} \to r'(t)

(componentwise, then the dot product is a limit by the limit laws of 5.1).

The second term has absolute value at most (bounded)

\times\ \lVert E(a, v(h)) \rVert \to 0

\frac{\lVert v(h) \rVert}{|h|} = \big\lVert\frac{v(h)}{h}\big\rVert \to \lVert r'(t) \rVert

is bounded, and

v(h) \to O

(since

r

is continuous at

t

), so

E(a, v(h)) \to 0

.If

v(h) = O

the term is

0

because

E(a, O) = 0

by convention.

g'(t) = \nabla f(r(t))\cdot r'(t)

\blacksquare

Sum of the two limits.

Worked example Lecture 22

Let

r(t) = (t, 1/t)

for

t > 0

and

f(x, y) = x^2 + y^2

. Find

g'(t)

for

g = f \circ r

, and interpret it.

Solution.

Gradient and velocity. $\nabla f = (2x, 2y)$ , which at $r(t)$ is $(2t, \frac2t)$ . $r'(t) = (1, -\frac1{t^2})$ . Differentiate each coordinate.
Dot. $g'(t) = 2t\cdot1 + \frac2t\cdot\big(-\frac1{t^2}\big) = 2t - \frac2{t^3}$ . The path rule.
Check directly. $g(t) = t^2 + t^{-2}$ , so $g'(t) = 2t - 2t^{-3}$ . Same answer, as it must be.
Interpret. $g'(t) = 0$ at $t = 1$ , the point $(1, 1)$ of the hyperbola closest to the origin; $g'(t) \to -\infty$ as $t \to 0^+$ and $+\infty$ as $t \to \infty$ . $g$ is the squared distance from the origin to the moving point: it falls, reaches its minimum $2$ at $(1, 1)$ , then rises.

Answer. $g'(t) = 2t - \dfrac2{t^3}$ , zero exactly when the path touches a level circle of $f$ tangentially (at $(1, 1)$ ).

Your answerFor

r(t) = (t, 1/t)

and

f = x^2 + y^2

, what is

\dfrac{d}{dt}f(r(t))

t = 2

Worked example MATH 375 HW 13 (Apostol 8.17 #1 and #2a)

With

u = f(x, y)

x = X(t)

y = Y(t)

and

F(t) = f(X(t), Y(t))

, express

F'

and

F''

; then evaluate for

f = x^2 + y^2

X = t

Y = t^2

Solution.

First derivative. By the path theorem with $r = (X, Y)$ : $F'(t) = f_xX' + f_yY'$ , the partials evaluated at $(X(t), Y(t))$ . Gradient dotted with velocity.
Second derivative: product rule. $F'' = \big(\tfrac{d}{dt}f_x\big)X' + f_xX'' + \big(\tfrac{d}{dt}f_y\big)Y' + f_yY''$ . Each product $f_x\cdot X'$ is a product of two functions of $t$ .
The partials are themselves composite. Apply the path theorem again, to the scalar fields $f_x$ and $f_y$ : $\tfrac{d}{dt}f_x = f_{xx}X' + f_{xy}Y'$ and $\tfrac{d}{dt}f_y = f_{yx}X' + f_{yy}Y'$ (with the order of differentiation as in 5.2). This is the step the homework warned about: $f_x(X(t), Y(t))$ also depends on $t$ .
Collect. If $f_{xy} = f_{yx}$ (true when they are continuous): $F'' = f_{xx}X'^2 + 2f_{xy}X'Y' + f_{yy}Y'^2 + f_xX'' + f_yY''$ .
The example. $f_x = 2x$ , $f_y = 2y$ , $f_{xx} = f_{yy} = 2$ , $f_{xy} = 0$ ; $X' = 1$ , $X'' = 0$ , $Y' = 2t$ , $Y'' = 2$ . So $F' = 2t\cdot1 + 2t^2\cdot2t = 2t + 4t^3$ and $F'' = 2\cdot1 + 2\cdot(2t)^2 + 0 + 2t^2\cdot2 = 2 + 12t^2$ . Check: $F = t^2 + t^4$ directly.

Answer. $F' = f_xX' + f_yY'$ ; $F'' = f_{xx}X'^2 + 2f_{xy}X'Y' + f_{yy}Y'^2 + f_xX'' + f_yY''$ ; for the example $F' = 2t + 4t^3$ and $F'' = 2 + 12t^2$ , as in Apostol’s answers.

Worked example MATH 375 Fall 2025 final, problem 5

A buoy at position

(X(t), Y(t))

measures a temperature

f(x, y, t)

that also changes in time. Find

\dfrac d{dt}f(X(t), Y(t), t)

; evaluate for

f = xt

and

(X, Y) = (t, 0)

Solution.

The curve in $(x, y, t)$ space. $r(t) = (X(t), Y(t), t)$ , with velocity $r'(t) = (X', Y', 1)$ . Time is also a coordinate; its velocity is $1$ .
Apply the path theorem. $g'(t) = f_xX' + f_yY' + f_t$ , all evaluated at $(X(t), Y(t), t)$ . Gradient in three variables dotted with $(X', Y', 1)$ .
Meaning. $f_xX' + f_yY'$ is the change due to the buoy’s motion; $f_t$ is the change that the temperature has at a fixed place even if the buoy sits still. Stand outside for a while and see the temperature change without moving.
The example. $f = xt$ : $f_x = t$ , $f_y = 0$ , $f_t = x$ . With $X = t$ , $Y = 0$ : $g' = t\cdot1 + 0 + t = 2t$ . Check: $g(t) = t\cdot t = t^2$ , $g' = 2t$ .

Answer. $g' = f_xX' + f_yY' + f_t$ . The forgotten term on the exam was $f_t$ .

Level sets, normals and tangent planes

Theorem T Apostol 8.16; Spagnolie Lecture 22

The gradient is perpendicular to level sets

Let $f$ be differentiable on an open set $S$ and let $r$ be a differentiable curve with $f(r(t)) = c$ for all $t$ near $t_0$ . Put $a = r(t_0)$ . Then $\nabla f(a)\cdot r'(t_0) = 0$ .

In one lineIf $f$ is differentiable at $a$ and $r$ is a curve in the level set $f = c$ through $a$ , then $\nabla f(a)\cdot r'(t_0) = 0$ : the gradient is perpendicular to the tangent vector of every such curve.

Why it is trueIt is the geometric content of 'steepest ascent': since $f$ does not change along a level curve, the direction in which it changes most must be at right angles to it.

ProofApostol 8.16; Lecture 22

The idea: The function g(t) = f(r(t)) is constant, so its derivative is 0; the path theorem says that derivative is the dot product.

g(t) = f(r(t))

is constant near

t_0

, so

g'(t_0) = 0

By hypothesis

f(r(t)) = c

Also

g'(t_0) = \nabla f(r(t_0))\cdot r'(t_0)

Rate of change along a curve.

\nabla f(a)\cdot r'(t_0) = 0

\blacksquare

Equate the two.

Definition T

Tangent plane

Let $a$ lie on the level surface $L(c) = \{f = c\}$ of a differentiable scalar field on $\mathbb R^3$ with $\nabla f(a) \ne O$ . By the theorem, the tangent vectors at $a$ of all curves lying in $L(c)$ are perpendicular to $\nabla f(a)$ , so they all lie in the plane through $a$ with normal vector $\nabla f(a)$ . (That they fill the whole plane, that is, that every direction in the plane is the velocity of some curve on the surface, is the content of the implicit function theorem, which comes later; Apostol assumes it in 8.16.) This plane is the tangent plane of the surface at $a$ :

\nabla f(a)\cdot(x - a) = 0, \quad\text{that is,}\quad D_1f(a)(x - a_1) + D_2f(a)(y - a_2) + D_3f(a)(z - a_3) = 0.

For a level curve of a field on $\mathbb R^2$ the same formula gives the tangent line. For the graph $z = f(x, y)$ , view it as the level set $F = 0$ of $F(x, y, z) = f(x, y) - z$ , with $\nabla F = (f_x, f_y, -1)$ : the tangent plane is $z = f(a_1, a_2) + \nabla f(a_1, a_2)\cdot\big((x, y) - (a_1, a_2)\big)$ , the graph of the linearization.

In one lineFor a level surface $f(x, y, z) = c$ and a point $a$ on it with $\nabla f(a) \ne O$ , the tangent plane is $\{x : \nabla f(a)\cdot(x - a) = 0\}$ ; in the plane the tangent line to a level curve is defined the same way.

Why it existsIt is the plane that contains the tangent vector of every curve on the surface through $a$ ; for a graph it is the plane of the linearization.

Quick checkFor

f(x, y) = xy

, the level curve through

(1, 1)

xy = 1

. In which direction does it run at

(1, 1)

Worked example Lecture 22

Find the tangent plane to

x^2 + y^2 + (z/2)^2 = \frac94

a = (1, 1, 1)

Solution.

Check that $a$ is on the surface. $1 + 1 + \frac14 = \frac94$ . $f(x, y, z) = x^2 + y^2 + (z/2)^2$ and $c = \frac94$ .
Gradient. $\nabla f = (2x, 2y, z/2)$ , so $\nabla f(a) = (2, 2, \frac12)$ . The normal vector.
Plane. $2(x - 1) + 2(y - 1) + \frac12(z - 1) = 0$ , that is $2x + 2y + \frac12z = \frac92$ . $2 + 2 + \frac12 = \frac92$ .

Answer. $2x + 2y + \frac12z = \frac92$ . (It is linear in $x, y, z$ ; if yours has squares, you made a slip, as Spagnolie said. And it does not pass through the origin, so it is not a subspace.)

Worked example Practice Final 2013, problem 5

Write an equation of the plane tangent to the surface

x^3y + z^2 = 3

a = (-1, 1, 2)

Solution.

Check. $(-1)^3\cdot1 + 4 = 3$ . $a$ lies on the surface.
Gradient. $\nabla(x^3y + z^2) = (3x^2y,\ x^3,\ 2z)$ , which at $a$ is $(3, -1, 4)$ .
Plane. $3(x + 1) - (y - 1) + 4(z - 2) = 0$ , that is $3x - y + 4z = 4$ . $-3 - 1 + 8 = 4$ is the check that $a$ satisfies the equation.

Answer. $3x - y + 4z = 4$ .

Worked example MATH 375 HW 13 (Apostol 8.17 #4)

Find a normal vector to the surface

z = \sqrt{x^2 + y^2} + (x^2 + y^2)^{3/2}

at a general point other than the origin, and the limit of the cosine of its angle with the

z

-axis as the point tends to the origin.

Solution.

Make it a level set. $G(x, y, z) = \sqrt{x^2 + y^2} + (x^2 + y^2)^{3/2} - z = 0$ . A graph $z = h(x, y)$ is the zero level set of $h - z$ .
Gradient. With $\rho = \sqrt{x^2 + y^2}$ : $G_x = \dfrac{x}{\rho} + 3\rho x = \dfrac{x(1 + 3\rho^2)}{\rho}$ , similarly $G_y = \dfrac{y(1 + 3\rho^2)}{\rho}$ , and $G_z = -1$ . $\frac{\partial\rho}{\partial x} = \frac x\rho$ (5.2).
Normal vector. $V = \nabla G = \dfrac{1 + 3\rho^2}{\rho}(x, y, 0) + (0, 0, -1)$ ; any scalar multiple will do. Normal to the tangent plane at each point.
Length. $\lVert(G_x, G_y)\rVert = \dfrac{(1 + 3\rho^2)}{\rho}\cdot\rho = 1 + 3\rho^2$ , so $\lVert V \rVert^2 = (1 + 3\rho^2)^2 + 1$ . $\lVert(x, y)\rVert = \rho$ .
Cosine with the $z$ -axis. $\cos\theta = \dfrac{|V\cdot\mathbf k|}{\lVert V \rVert} = \dfrac1{\sqrt{(1 + 3\rho^2)^2 + 1}}$ (taking the angle with the upward normal), which tends to $\dfrac1{\sqrt2}$ as $\rho \to 0$ . Set $\rho = 0$ .

Answer. $V = \Big(\dfrac{(1 + 3\rho^2)x}{\rho},\ \dfrac{(1 + 3\rho^2)y}{\rho},\ -1\Big)$ and $\cos\theta \to \dfrac1{\sqrt2}$ , so $\theta \to \frac\pi4$ : near the origin the surface is cone-like, with slope $45^\circ$ (it behaves like $z \approx \rho$ ).

Worked example MATH 375 HW 13 (Apostol 8.17 #8); the printed key writes $a^2$ for $a^3$ by mistake

Find the tangent plane to

xyz = a^3

(x_0, y_0, z_0)

and show the tetrahedron it cuts from the first octant has volume

\frac92a^3

Solution.

Gradient. $\nabla(xyz) = (yz, xz, xy)$ , so at the point it is $(y_0z_0,\ x_0z_0,\ x_0y_0)$ . The normal vector.
Plane. $y_0z_0(x - x_0) + x_0z_0(y - y_0) + x_0y_0(z - z_0) = 0$ . Expand and use $x_0y_0z_0 = a^3$ : $y_0z_0x + x_0z_0y + x_0y_0z = 3x_0y_0z_0$ . Each of the three terms contributes $-x_0y_0z_0$ on the right.
Divide by $x_0y_0z_0$ . $\dfrac x{x_0} + \dfrac y{y_0} + \dfrac z{z_0} = 3$ . The intercept form of a plane.
Intercepts. $(3x_0, 0, 0)$ , $(0, 3y_0, 0)$ , $(0, 0, 3z_0)$ . Set two of $x, y, z$ to $0$ .
Volume. $\frac13\cdot\text{base}\cdot\text{height} = \frac13\cdot\frac12(3x_0)(3y_0)\cdot3z_0 = \frac{27}6x_0y_0z_0 = \frac92a^3$ . A tetrahedron with three mutually perpendicular edges of lengths $p, q, s$ has volume $\frac{pqs}6$ .

Answer. $\frac x{x_0} + \frac y{y_0} + \frac z{z_0} = 3$ ; the volume is $\frac92a^3$ , independent of the point.

Worked example

Find the tangent plane to the graph

z = e^x\cos y

at the point

(0, 0, 1)

Solution.

Check the point. $e^0\cos 0 = 1$ , so $(0, 0, 1)$ is on the graph. The third coordinate must equal $f(0, 0)$ .
Partials of $f(x, y) = e^x\cos y$ . $f_x = e^x\cos y$ and $f_y = -e^x\sin y$ , which at $(0, 0)$ are $1$ and $0$ . Both partials are continuous, so $f$ is differentiable there and the linearization is the tangent plane.
Use the graph formula. $z = f(0, 0) + f_x(0, 0)\,(x - 0) + f_y(0, 0)\,(y - 0) = 1 + x$ . The tangent plane of a graph is the graph of the linearization.
Cross-check as a level surface. With $G(x, y, z) = e^x\cos y - z$ we have $\nabla G(0, 0, 1) = (1, 0, -1)$ , and $1\cdot x + 0\cdot y - 1\cdot(z - 1) = 0$ gives the same plane $z = 1 + x$ . The two descriptions of a tangent plane agree, as the definition promised.

Answer. $z = 1 + x$ , that is $x - z = -1$ .

Worked example Apostol 8.17 #3(a); the printed answer is $-\frac23$

Find the directional derivative of

f(x, y, z) = 3x - 5y + 2z

(2, 2, 1)

in the direction of the outward normal to the sphere

x^2 + y^2 + z^2 = 9

Solution.

Check the point is on the sphere. $4 + 4 + 1 = 9$ . Otherwise “the normal to the sphere at the point” would make no sense.
Find the normal. The sphere is the level set $g = 9$ of $g(x, y, z) = x^2 + y^2 + z^2$ , and $\nabla g = (2x, 2y, 2z)$ is $(4, 4, 2)$ at the point. The gradient is perpendicular to the level surface. It points outward because $g$ grows as we move away from the origin.
Make it a unit vector. $\lVert(4, 4, 2)\rVert = \sqrt{36} = 6$ , so $u = \dfrac{(4, 4, 2)}{6} = \dfrac{(2, 2, 1)}{3}$ . A directional derivative needs a unit vector. Here the outward normal is the position vector divided by the radius $3$ .
Gradient of $f$ . $f$ is linear, so $\nabla f = (3, -5, 2)$ at every point. The linear field $a\cdot x$ has gradient $a$ (example in 5.2).
Dot. $f'(a; u) = \nabla f\cdot u = \dfrac{3\cdot2 - 5\cdot2 + 2\cdot1}{3} = \dfrac{-2}{3}$ . Differentiable, so $f'(a; u) = \nabla f(a)\cdot u$ .

Answer. $-\dfrac23$ , which is the answer printed at the back of the book. The negative sign says $f$ decreases as we leave the sphere in the outward direction.

Worked example Apostol 8.17 #10, with new numbers

Find

c

so that the spheres

x^2 + y^2 + z^2 = 4

and

(x - c)^2 + y^2 + z^2 = 1

meet at right angles: at every common point their tangent planes are perpendicular.

Solution.

Normals. Take the spheres as level sets. The normal of the first at $(x, y, z)$ is $V_1 = 2(x, y, z)$ , and of the second $V_2 = 2(x - c, y, z)$ . Gradients of the two defining expressions.
Perpendicular planes. Two planes are perpendicular exactly when their normals are, so we need $V_1\cdot V_2 = 0$ : $x(x - c) + y^2 + z^2 = 0$ , that is $(x^2 + y^2 + z^2) - cx = 0$ . Dot product of the normals.
Use the first sphere. $x^2 + y^2 + z^2 = 4$ , so the condition is $4 - cx = 0$ , that is $cx = 4$ at every common point. The point is on the first sphere.
Use the second sphere. Expanding, $x^2 + y^2 + z^2 - 2cx + c^2 = 1$ , so $4 - 2cx + c^2 = 1$ . With $cx = 4$ this is $4 - 8 + c^2 = 1$ . The point is also on the second sphere, and step 3 gives the value of $cx$ .
Solve. $c^2 = 5$ , so $c = \pm\sqrt5$ . Check with geometry: spheres meet at right angles when the distance between centres satisfies $d^2 = r_1^2 + r_2^2$ , here $c^2 = 4 + 1$ .

Answer. $c = \pm\sqrt5$ .

Worked example Apostol 8.17 #9, with new surfaces

Find a pair of linear equations for the line tangent to both surfaces

x^2 + y^2 + z^2 = 3

and

z = xy

at the point

(1, 1, 1)

Solution.

Check the point. $1 + 1 + 1 = 3$ and $1\cdot1 = 1$ . It lies on both surfaces.
First tangent plane. $\nabla(x^2 + y^2 + z^2) = (2, 2, 2)$ at the point, so the plane is $2(x - 1) + 2(y - 1) + 2(z - 1) = 0$ , that is $x + y + z = 3$ . Tangent plane of a level surface.
Second tangent plane. Write $z = xy$ as the level set of $G = xy - z$ at $0$ : $\nabla G = (y, x, -1) = (1, 1, -1)$ , so $(x - 1) + (y - 1) - (z - 1) = 0$ , that is $x + y - z = 1$ . The graph is a level set of $f(x, y) - z$ .
The common tangent line lies in both planes. Its equations are the two plane equations together. A line tangent to both surfaces has its direction perpendicular to both normals, so it lies in both tangent planes.
Check the direction. A direction $(p, q, r)$ perpendicular to both normals needs $p + q + r = 0$ and $p + q - r = 0$ , so $r = 0$ and $q = -p$ : the line runs in the direction $(-1, 1, 0)$ . Moving by $(-1, 1, 0)$ keeps both $x + y + z$ and $x + y - z$ fixed, so we stay in both planes.

Answer. $x + y + z = 3$ and $x + y - z = 1$ (the line through $(1, 1, 1)$ with direction $(-1, 1, 0)$ ).

Watch out· Partial derivatives are not enough for a tangent plane

For $f = \sqrt{|xy|}$ both partial derivatives are $0$ at the origin, which suggests the tangent plane $z = 0$ . But cutting the graph by the plane $x = y$ gives $z = |x|$ , which has a corner (5.2, stretch problem; Apostol 8.17 #6). The surface has no tangent plane at the origin because $f$ is not differentiable there. The tangent plane exists exactly when the field is differentiable.

Vector fields and the Jacobian at work

Worked example Lecture 23 (polar coordinates); Jacobi 1841

Find the Jacobian matrix of the polar-coordinate map

P(r, \theta) = (r\cos\theta,\ r\sin\theta)

and its determinant.

Solution.

Components. $x = r\cos\theta$ , $y = r\sin\theta$ . $P: \mathbb R^2 \to \mathbb R^2$ , so the Jacobian is $2 \times 2$ .
Partials. $\dfrac{\partial x}{\partial r} = \cos\theta$ , $\dfrac{\partial x}{\partial\theta} = -r\sin\theta$ , $\dfrac{\partial y}{\partial r} = \sin\theta$ , $\dfrac{\partial y}{\partial\theta} = r\cos\theta$ .
Matrix. $J_P = \begin{bmatrix}\cos\theta & -r\sin\theta\\ \sin\theta & r\cos\theta\end{bmatrix}$ . Row 1: gradient of $x$ ; row 2: gradient of $y$ .
Determinant. $\det J_P = r\cos^2\theta + r\sin^2\theta = r$ . $\cos^2 + \sin^2 = 1$ .

Answer. $\det J_P = r$ . A small rectangle of sides $dr$ and $d\theta$ is sent to a patch of area about $r\,dr\,d\theta$ : the factor $r$ in the area element of polar coordinates. For example the quarter-annulus $1 \le r \le 2$ , $0 \le \theta \le \frac\pi2$ has area $\int\!\!\int r\,dr\,d\theta = \frac\pi2\cdot\frac{4 - 1}2 = \frac{3\pi}4$ , which is $\frac14\pi(2^2 - 1^2)$ .

Your answerThe Jacobian determinant of the polar-coordinate map is

r

. At the point

r = 3

, by what factor are small areas multiplied?

Worked example MATH 375 HW 13 (Apostol 8.22 #13)

(a) If

f(x, y, z) = (x, y, z)

, show

Df = I

. (b) Find all differentiable

f: \mathbb R^3 \to \mathbb R^3

with

Df = I

. (c) Find all those with

Df

diagonal,

\operatorname{diag}(p(x), q(y), r(z))

Solution.

(a) $(Df)_{ij} = D_jf_i$ with $f_1 = x$ , $f_2 = y$ , $f_3 = z$ : $D_jf_i = 1$ if $i = j$ and $0$ otherwise. So $Df = I$ . The identity field has the identity matrix as its derivative, as a linear map is its own derivative.
(b) Row by row. $Df = I$ means $D_1f_1 = 1$ , $D_2f_1 = D_3f_1 = 0$ . The last two say $f_1$ does not depend on $y$ or $z$ (a field with $D_2f_1 = D_3f_1 = 0$ is constant in those variables on a ball, by the mean value theorem of 5.2), so $f_1 = g(x)$ ; then $g' = 1$ , $g = x + c_1$ . Integrate the one-variable equation.
(b) The other rows. Likewise $f_2 = y + c_2$ and $f_3 = z + c_3$ . So $f(x, y, z) = (x, y, z) + c$ for a constant vector $c$ . Translations are the only fields whose derivative is the identity everywhere.
(c) Off-diagonal entries $0$ mean $f_1$ depends only on $x$ , $f_2$ only on $y$ , $f_3$ only on $z$ , with $f_1' = p$ , $f_2' = q$ , $f_3' = r$ . So $f = (P(x),\ Q(y),\ R(z))$ with $P' = p$ , $Q' = q$ , $R' = r$ : antiderivatives, each up to a constant. The key stops at “f_i depends on one variable”; the last step is the fundamental theorem of calculus (1.5).

Answer. (a) $Df = I$ . (b) $f(x) = x + c$ . (c) $f(x, y, z) = (P(x), Q(y), R(z))$ where $P, Q, R$ are antiderivatives of $p, q, r$ .

Same idea elsewhere· Lecture 19's fluid flow was a Jacobian all along

In the first Lecture 19, the velocity of a steady linear flow is $u(x) = xA$ for a matrix $A$ (a row vector times $A$ ). As a column, $u = A^{\mathsf T}x$ , a linear field, so by the example above its Jacobian is the constant matrix $A^{\mathsf T}$ . Splitting $A$ into its symmetric and antisymmetric parts $E + R$ is splitting the Jacobian of the velocity field into stretching (the symmetric part, whose eigenvectors are the axes along which a ball of fluid is stretched or squeezed) and rotation (the antisymmetric part). The picture above, with the field $(2x + y, x + 2y)$ , is exactly the lecture’s example: $A = \begin{bmatrix}2 & 1\\ 1 & 2\end{bmatrix}$ , eigenvalues $3$ and $1$ , directions $(1, 1)$ and $(1, -1)$ . A circle of fluid becomes an ellipse stretched most along the diagonal.

Where it is used

Used inside maths· Inside maths: the chain rule, the inverse function theorem, and changes of variables

The next chapter (5.4) proves the chain rule $D(F \circ G)(a) = DF(G(a))\,DG(a)$ : the Jacobian of a composition is the product of the Jacobians, with the same proof idea as the one-variable chain rule of 1.4 now that “derivative” means “linear map”. The inverse function theorem says a $C^1$ map $F: \mathbb R^n \to \mathbb R^n$ with $\det DF(a) \ne 0$ is invertible near $a$ : a map that is locally a good linear map with an invertible matrix is locally invertible. The determinant is the factor in the change of variables formula for integrals (the $r$ of polar coordinates). Total derivatives also give the second-order Taylor formula (the Hessian, 5.6), and the second-derivative test combines this with the eigenvalues of 3.1 and 3.2.

Used in the real world· Robotics, fluids and AI: the Jacobian as 'speed in, speed out'

A robot arm with two links of length 1 m has joint angles $\theta_1, \theta_2$ and hand position $(x, y) = (\cos\theta_1 + \cos(\theta_1 + \theta_2),\ \sin\theta_1 + \sin(\theta_1 + \theta_2))$ . Its Jacobian is

J = \begin{bmatrix}-\sin\theta_1 - \sin(\theta_1 + \theta_2) & -\sin(\theta_1 + \theta_2)\\ \cos\theta_1 + \cos(\theta_1 + \theta_2) & \cos(\theta_1 + \theta_2)\end{bmatrix},\qquad \det J = \sin\theta_2.

At $\theta_1 = 0$ , $\theta_2 = \frac\pi2$ the matrix is $\begin{bmatrix}-1 & -1\\ 1 & 0\end{bmatrix}$ and joint speeds $(0.1, 0.2)$ rad/s move the hand at $J\,(0.1, 0.2) = (-0.3, 0.1)$ m/s. When the arm is stretched out straight ( $\theta_2 = 0$ ), $\det J = 0$ : the hand cannot move along the arm, and no joint speeds produce such a motion. These are the singular positions that robot controllers avoid, found by solving $\det J = 0$ .

In fluid mechanics (Lecture 19) the Jacobian of the velocity field is the velocity gradient: its trace is the rate at which volume grows, and for $u = xA$ with $A = \begin{bmatrix}2&1&0\\1&2&0\\0&0&0\end{bmatrix}$ (eigenvalues $3, 1, 0$ ) a unit ball of fluid becomes, after time $t$ , an ellipsoid with semi-axes $e^{3t}, e^{t}, 1$ ; at $t = 0.5$ they are $4.48, 1.65, 1$ and the volume has grown by $e^{4t} = e^2 \approx 7.39$ . In machine learning, a layer $h \mapsto \sigma(Wh + b)$ is a differentiable map whose Jacobian is $\operatorname{diag}(\sigma')\,W$ ; backpropagation (3.8) multiplies these Jacobians from the loss backwards.

Traps

Watch out· The derivative is a map, not a number

For $F: \mathbb R^n \to \mathbb R^m$ the derivative at $a$ is a linear map (an $m \times n$ matrix of numbers). The gradient $\nabla f(a)$ is a column ( $n \times 1$ ), the Jacobian of the same $f$ is the row $\nabla f(a)^{\mathsf T}$ ( $1 \times n$ ). Check shapes before multiplying.

Watch out· Existing partial derivatives do not give the total derivative

Having $\nabla f(a)$ is not the same as being differentiable. Differentiability needs the error to be small along every path. Continuous partials near $a$ is a quick sufficient test; it is not necessary, and failing it proves nothing.

Watch out· Non-smooth level sets do not mean non-differentiable

Fall 2025 final, true or false: “if the level sets of $f$ are not smooth then $f$ is not differentiable” is false. Take $f(x) = e^{-1/x}$ for $x > 0$ and $f(x) = 0$ for $x \le 0$ , infinitely differentiable, whose zero level set is a half-line that stops abruptly; and $f(x)f(y)$ has level sets with corners. The theorem that level sets are smooth surfaces needs $\nabla f \ne O$ (the implicit function theorem), and the theorem says nothing when $\nabla f = O$ .

Watch out· Divide by the length of the step, not by the step

The error condition is $\frac{\lVert\text{error}\rVert}{\lVert v \rVert} \to 0$ . You cannot divide by a vector. A bound such as “error $\le C\lVert v \rVert^2$ ” suffices.

Watch out· Do not forget the explicit time variable

If the field depends on time as well as position, the derivative along the path has an extra term $f_t$ (the buoy on the final). The velocity of the curve $(X(t), Y(t), t)$ in space-time has a last entry $1$ .

Watch out· A tangent plane needs a nonzero gradient

If $\nabla f(a) = O$ then $\nabla f(a)\cdot(x - a) = 0$ is not a plane. The level set can have a corner or a crossing there: the level set of $x^2 - y^2$ through the origin is two crossing lines.

Problems

check

State $F'(a)$ (as a matrix) for $F(x, y) = (3x - y,\ 2y,\ x + 5y)$ .

Solution

$F$ is linear, $F(x) = Ax$ with $A = \begin{bmatrix}3 & -1\\ 0 & 2\\ 1 & 5\end{bmatrix}$ , so $F'(a) = A$ for every $a$ . It is $3 \times 2$ : three outputs, two inputs.

check

Find the Jacobian of $F(x, y) = (x^2y,\ x + y)$ at $(1, 2)$ and the linearization there.

Solution

$DF = \begin{bmatrix}2xy & x^2\\ 1 & 1\end{bmatrix}$ , so $DF(1, 2) = \begin{bmatrix}4 & 1\\ 1 & 1\end{bmatrix}$ and $F(1, 2) = (2, 3)$ . $L(x, y) = \big(2 + 4(x - 1) + (y - 2),\ 3 + (x - 1) + (y - 2)\big)$ .

check

Find the tangent plane to the graph $z = x^2 + y^2$ at $(1, 2, 5)$ .

Solution

$f = x^2 + y^2$ , $\nabla f(1, 2) = (2, 4)$ , so $z = 5 + 2(x - 1) + 4(y - 2)$ , that is $2x + 4y - z = 5$ .

checkMATH 375 HW 12 (Apostol 8.14 #11)

True or false: if $f$ has continuous partial derivatives near $a$ , then $f$ is continuous at $a$ . Is the converse true?

Solution

True: continuous partials imply differentiable, and differentiable implies continuous. The converse is false: $\lVert x \rVert$ is continuous at the origin but its partials do not exist there.

useMATH 375 HW 13 (Apostol 8.17 #10)

Find the constant $c$ such that at any intersection point of the spheres $(x - c)^2 + y^2 + z^2 = 3$ and $x^2 + (y - 1)^2 + z^2 = 1$ the tangent planes are perpendicular.

Solution

Normals: $V_1 = 2(x - c, y, z)$ and $V_2 = 2(x, y - 1, z)$ . The planes are perpendicular exactly when $V_1\cdot V_2 = 0$ , that is $x(x - c) + y(y - 1) + z^2 = 0$ . Expanding the second sphere, $x^2 + y^2 - 2y + 1 + z^2 = 1$ , gives $x^2 + y^2 + z^2 = 2y$ . Substituting this into the condition: $2y - cx - y = 0$ , so $y = cx$ . So every intersection point must lie on the plane $y = cx$ . Now expand the first sphere, $x^2 - 2cx + c^2 + y^2 + z^2 = 3$ , and substitute $x^2 + y^2 + z^2 = 2y$ : $2y - 2cx + c^2 = 3$ . With $y = cx$ the $x$ -terms cancel and $c^2 = 3$ , so $c = \pm\sqrt3$ . (Check: two spheres meet at right angles exactly when $d^2 = r_1^2 + r_2^2$ for the distance $d$ between their centres $(c, 0, 0)$ and $(0, 1, 0)$ ; here $c^2 + 1 = 3 + 1$ . Apostol’s answer is the same.)

useApostol 8.17 #9

Find a pair of linear equations for the line tangent to both surfaces $x^2 + y^2 + 2z^2 = 4$ and $z = e^{x - y}$ at the point $(1, 1, 1)$ .

Solution

Both surfaces pass through $(1, 1, 1)$ ( $1 + 1 + 2 = 4$ and $e^0 = 1$ ). First surface $G_1 = x^2 + y^2 + 2z^2$ : $\nabla G_1 = (2x, 2y, 4z) = (2, 2, 4)$ , tangent plane $2(x - 1) + 2(y - 1) + 4(z - 1) = 0$ , that is $x + y + 2z = 4$ . Second surface $G_2 = z - e^{x - y}$ : $\nabla G_2 = (-e^{x - y}, e^{x - y}, 1) = (-1, 1, 1)$ , tangent plane $-(x - 1) + (y - 1) + (z - 1) = 0$ , that is $-x + y + z = 1$ . The common tangent line lies in both planes: $x + y + 2z = 4$ , $-x + y + z = 1$ .

useApostol 8.17 #7

Show that the tangent plane to the surface $z = xy$ at a point $(x_0, y_0, z_0)$ contains the two lines $y = y_0,\ z = y_0x$ and $x = x_0,\ z = x_0y$ .

Solution

Plane: $z = x_0y_0 + y_0(x - x_0) + x_0(y - y_0)$ (graph of the linearization, $\nabla(xy) = (y, x)$ ). The first line has points $(x, y_0, y_0x)$ : substituting, the right side is $x_0y_0 + y_0(x - x_0) + 0 = y_0x$ , which equals $z$ . The second line has points $(x_0, y, x_0y)$ : the right side is $x_0y_0 + 0 + x_0(y - y_0) = x_0y$ , equal to $z$ . Both lines lie in the plane (and on the surface).

useApostol 8.17 #3(b)

Find the directional derivative of $f(x, y, z) = x^2 - y^2$ at a general point of the sphere $x^2 + y^2 + z^2 = 4$ , in the direction of the outward normal at that point.

Solution

The outward normal is $\nabla(x^2 + y^2 + z^2) = 2(x, y, z)$ , of length $2\sqrt{x^2 + y^2 + z^2} = 4$ on the sphere, so $u = \frac{(x, y, z)}{2}$ . Also $\nabla f = (2x, -2y, 0)$ . So $f'(a; u) = \frac{2x\cdot x - 2y\cdot y + 0}{2} = x^2 - y^2$ , which is Apostol’s answer.

stretchApostol 8.17 #3(c)

Find the directional derivative of $f(x, y, z) = x^2 + y^2 - z^2$ at $(3, 4, 5)$ along the curve of intersection of the surfaces $2x^2 + 2y^2 - z^2 = 25$ and $x^2 + y^2 = z^2$ .

Solution

Both surfaces contain $(3, 4, 5)$ : $18 + 32 - 25 = 25$ and $9 + 16 = 25$ . The curve lies on both, so its tangent is perpendicular to both gradients (a curve in a level set has velocity perpendicular to the gradient). The normals are $N_1 = (4x, 4y, -2z) = (12, 16, -10)$ and $N_2 = (2x, 2y, -2z) = (6, 8, -10)$ . We need $T = (p, q, r)$ with $N_1\cdot T = 12p + 16q - 10r = 0$ and $N_2\cdot T = 6p + 8q - 10r = 0$ . Subtracting gives $6p + 8q = 0$ , so $(p, q) = t(-4, 3)$ , and then $r = 0$ . So the unit tangent is $\pm(-4, 3, 0)/5$ . Now $\nabla f = (2x, 2y, -2z) = (6, 8, -10)$ , and $\nabla f\cdot(-4, 3, 0)/5 = \frac{-24 + 24}{5} = 0$ . The answer is $0$ , as in Apostol. (The reason: $f = x^2 + y^2 - z^2$ is $0$ on the second surface, so $f$ is constant along the curve.)

use

Using the Jacobian of the polar map, find the area of the region $\{1 \le x^2 + y^2 \le 4,\ x \ge 0,\ y \ge 0\}$ . How does the result relate to the area of a full disc of radius 2?

Solution

In polar coordinates the region is $1 \le r \le 2$ , $0 \le \theta \le \frac\pi2$ and the area element is $r\,dr\,d\theta$ (the Jacobian determinant). Area $= \int_0^{\pi/2}\int_1^2r\,dr\,d\theta = \frac\pi2\cdot\frac{4 - 1}2 = \frac{3\pi}4$ . A full disc of radius 2 has area $4\pi$ ; the region is the quarter ( $\pi$ ) of the disc minus the quarter of the unit disc ( $\frac\pi4$ ): $\pi - \frac\pi4 = \frac{3\pi}4$ . Same.

prove

Prove that if $f$ and $g$ are differentiable at $a$ with total derivatives $T_a$ and $S_a$ , then $f + g$ is differentiable at $a$ with total derivative $T_a + S_a$ .

Solution

Plan. Add the two defining equations.

$f(a + v) = f(a) + T_a(v) + \lVert v \rVert E_f$ and $g(a + v) = g(a) + S_a(v) + \lVert v \rVert E_g$ with $E_f, E_g \to 0$ . Adding: $(f + g)(a + v) = (f + g)(a) + (T_a + S_a)(v) + \lVert v \rVert(E_f + E_g)$ . The sum of two linear maps is linear and $E_f + E_g \to 0$ . $\blacksquare$

proveMATH 375 HW 12 (Apostol 8.14 #10a) via 5.2

Prove: if $f$ is differentiable at every point of a ball $B(a, r)$ and $\nabla f(x) = O$ for every $x$ in the ball, then $f$ is constant on the ball.

Solution

Plan. A differentiable field has $f'(x; y) = \nabla f(x)\cdot y$ ; here that is $0$ for every $y$ , so the result of 5.2 applies.

By the dot-product theorem, $f'(x; y) = \nabla f(x)\cdot y = 0$ for all $x$ in the ball and all $y$ . By the problem of 5.2 (mean value theorem for scalar fields applied on the segment from $a$ to $x$ , which lies in the ball), $f(x) - f(a) = f'(z; x - a) = 0$ . So $f(x) = f(a)$ for every $x$ in the ball. $\blacksquare$

proveApostol 8.17 #11

If $r_1, r_2$ are the distances from a point $(x, y)$ of an ellipse to its foci $F_1, F_2$ , then $r_1 + r_2$ is constant. Show that the unit tangent $T$ satisfies $T\cdot\nabla(r_1 + r_2) = 0$ , and deduce that the tangent makes equal angles with the lines to the two foci.

Solution

Plan. The ellipse is a level curve of $r_1 + r_2$ ; the gradient of a distance is a unit vector pointing away from the focus.

The ellipse is the level set $r_1 + r_2 = \text{const}$ , so by the theorem of this chapter the gradient is perpendicular to the tangent: $T\cdot\nabla(r_1 + r_2) = 0$ . By Apostol 8.14 #8 (5.2), $\nabla r_i = u_i$ , the unit vector from the focus $F_i$ to the point. So $T\cdot u_1 = -T\cdot u_2$ , that is $\cos\alpha_1 = -\cos\alpha_2$ for the angles $\alpha_i$ between $T$ and $u_i$ . The angle between the tangent line and the segment from the point back to focus 1 equals the angle between the tangent line and the segment back to focus 2. This is the reflection property: light from one focus bounces off the ellipse to the other. $\blacksquare$

proveApostol 8.17 #12

If $\nabla f(x, y, z)$ is always parallel to $\mathbf r = (x, y, z)$ , show that $f(0, 0, a) = f(0, 0, -a)$ .

Solution

Plan. Walk along a half-circle on the sphere of radius $a$ from $(0, 0, a)$ to $(0, 0, -a)$ and show $f$ does not change.

Let $r(t) = (a\sin t, 0, a\cos t)$ , $0 \le t \le \pi$ . Then $\nabla f(r(t)) = \lambda(t)\,r(t)$ for some scalar $\lambda(t)$ (parallel to the position vector), and $r'(t) = (a\cos t, 0, -a\sin t)$ . By the path theorem $\frac d{dt}f(r(t)) = \lambda(t)\,r(t)\cdot r'(t) = \lambda(t)\big(a^2\sin t\cos t - a^2\cos t\sin t\big) = 0$ . So $f(r(t))$ is constant on $[0, \pi]$ (1.4), and $f(r(0)) = f(0, 0, a) = f(r(\pi)) = f(0, 0, -a)$ . $\blacksquare$

proveMATH 375 HW 12 style (Apostol 8.14 #11, the non-implication (b) does not imply (d))

Prove that $f(x, y) = (x^2 + y^2)\sin\dfrac1{x^2 + y^2}$ (with $f(0, 0) = 0$ ) is differentiable at the origin but its partial derivatives are not continuous there.

Solution

Plan. Differentiability by the size of $f$ ; discontinuity by the explicit formula for $f_x$ on the axis.

Since $|\sin| \le 1$ , $|f(v)| \le \lVert v \rVert^2$ . So $f(v) = f(0) + 0\cdot v + \lVert v \rVert E(v)$ with $E(v) = \lVert v \rVert\sin\frac1{\lVert v \rVert^2}$ , and $|E| \le \lVert v \rVert \to 0$ : differentiable at $O$ with $T_O = 0$ , so $\nabla f(O) = O$ . For $(x, y) \ne O$ , with $\rho^2 = x^2 + y^2$ : $f_x = 2x\sin\frac1{\rho^2} - \frac{2x}{\rho^2}\cos\frac1{\rho^2}$ . On the $x$ -axis at $x_n = (2\pi n)^{-1/2}$ : $\sin\frac1{x_n^2} = \sin 2\pi n = 0$ and $\cos 2\pi n = 1$ , so $f_x(x_n, 0) = -\frac2{x_n} \to -\infty$ , not tending to $f_x(O) = 0$ . So $f_x$ is not continuous at the origin. $\blacksquare$

stretch

Show that $F(x, y) = (x^2 - y^2,\ 2xy)$ has Jacobian determinant $4(x^2 + y^2)$ and that $J_F$ is always a scalar multiple of a rotation matrix. (Hint: compare $F$ with $z \mapsto z^2$ for $z = x + iy$ in the complex numbers of 0.6.)

Solution

$J_F = \begin{bmatrix}2x & -2y\\ 2y & 2x\end{bmatrix}$ and $\det J_F = 4x^2 + 4y^2$ . Writing $2\sqrt{x^2 + y^2} = s$ , $J_F = s\begin{bmatrix}\cos\phi & -\sin\phi\\ \sin\phi & \cos\phi\end{bmatrix}$ with $(\cos\phi, \sin\phi) = (x, y)/\sqrt{x^2 + y^2}$ : a rotation by $\phi$ scaled by $s$ . So $F$ maps every tiny circle to a tiny circle, rotated by the argument of $z$ and scaled by $2|z|$ : this is the derivative of $z^2$ , which is multiplication by $2z$ . The map preserves angles (is conformal), a property that holds for every complex-differentiable function.

stretchPractice Final 2013, extra credit

Is every $C^1$ vector field $g: \mathbb R^2 \to \mathbb R^2$ the gradient of some scalar field? Show that $g(x, y) = (-y, x)$ is not a gradient of any twice continuously differentiable $f$ .

Solution

If $g = \nabla f$ with $f$ having continuous second partials, then $g_1 = f_x$ , $g_2 = f_y$ , so $D_2g_1 = f_{xy}$ and $D_1g_2 = f_{yx}$ , which are equal for continuous second partials (the equality theorem of 5.2’s box). For $g = (-y, x)$ : $D_2g_1 = -1$ but $D_1g_2 = +1$ . They differ, so no such $f$ exists. The condition $D_jg_i = D_ig_j$ is necessary; the field $(-y, x)$ is the rotation field of the first picture, and going once around the origin with it never returns to the same “height”, which is why it cannot be the gradient of a function.

Summary

A field $F$ is differentiable at $a$ if $F(a + v) = F(a) + T_a(v) + \lVert v \rVert E(a, v)$ with $T_a$ linear and $E \to 0$ as $v \to 0$ . The linear map $T_a$ is the total derivative $F'(a)$ : the best linear approximation, with an error that is small compared with $\lVert v \rVert$ .
If $f$ is differentiable, then $f'(a; y)$ exists for every $y$ and $f'(a; y) = T_a(y) = \nabla f(a)\cdot y$ . So $T_a$ is unique and the gradient formula of 5.2 is a theorem. Differentiable fields are continuous.
A vector field is differentiable exactly when its components are, and the matrix of $F'(a)$ is the Jacobian matrix $J_F(a)$ with $(i, j)$ entry $D_jF_i(a)$ : rows are gradients, columns are $F'(a; e_j)$ . For a scalar field it is the row $\nabla f^{\mathsf T}$ .
Continuous partial derivatives near $a$ imply differentiable at $a$ (Apostol 8.13, proved by moving one coordinate at a time with the one-variable mean value theorem). The converse fails. Chain of implications: $C^1 \Rightarrow$ differentiable $\Rightarrow$ continuous and all directional derivatives exist; no arrow reverses (the table of HW 12).
The linearization $L(x) = F(a) + F'(a)(x - a)$ is the best affine approximation; for a scalar field its graph is the tangent plane $z = f(a) + \nabla f(a)\cdot(x - a)$ .
Along a curve, $\frac d{dt}f(r(t)) = \nabla f(r(t))\cdot r'(t)$ . Hence the gradient is perpendicular to every level set, and the tangent plane to $f = c$ at $a$ is $\nabla f(a)\cdot(x - a) = 0$ (when $\nabla f(a) \ne O$ ).
The Jacobian determinant is the local area or volume scaling (the $r$ of polar coordinates); the chain rule of 5.4 multiplies Jacobians.

Watch and read

The same ideas explained by someone else. Use these when an explanation here does not land.

5.3 The total derivative, the Jacobian matrix and tangent planes

The picture: zoom until it is flat

The idea, in words

The definition

Differentiable field

Total derivative

What the total derivative contains

Directional derivative as a dot product with the gradient

A vector field is differentiable exactly when its components are

Jacobian matrix

Differentiable implies continuous

Differentiable fields are continuous

When does the total derivative exist?

Continuously differentiable field

Continuous partial derivatives imply differentiable

The implication table of homework 12

The linearization and its error

Linearization

The derivative along a curve

Rate of change along a curve

Level sets, normals and tangent planes

The gradient is perpendicular to level sets

Tangent plane

Vector fields and the Jacobian at work

Where it is used

Traps

Problems

Summary

Watch and read

Where this sits

Comes from

Leads to

Same idea elsewhere