7.1 Existence and uniqueness for differential equations

A differential equation with a starting value has exactly one solution, for a short time, as long as its right-hand side does not stretch distances too fast. The proof turns the equation into an integral equation and applies the contraction mapping theorem to the space of continuous functions.

A population grows at a rate proportional to its size. A ball falls with an acceleration that depends on its speed. A capacitor drains at a rate that depends on its charge. Each of these gives an equation in which the unknown is a function and the equation involves the function’s derivative. Before we try to find a solution we should ask two questions that school courses skip. Is there a solution at all? And if there is, is there only one? The end of 4.2 gave a preview: turn the differential equation into an integral equation, and a solution becomes a fixed point of a contraction on a space of functions. This chapter carries that out in general.

Before you start

You need three earlier ideas: the contraction mapping theorem and the sup metric from 4.2, the fundamental theorem of calculus from 1.5, and the idea of a norm from 2.9 and 4.1. Three quick checks:

Quick checkWhich of these is NOT a hypothesis of the Banach fixed point theorem?

Quick checkOn

[0, 1]

let

f(t) = t

and

g(t) = t^2

. What is the sup distance

d_\infty(f, g) = \max_t \lvert f(t) - g(t) \rvert

Quick checkA continuous function

y

satisfies

y(t) = 1 + \int_0^t y(s)\,ds

for all

t

. What is

y'(t)

The picture

Each curve below is a Picard iterate. Start with the constant function $y_0 = 1$ , which is the starting value. Plug it into the right-hand side of the equation and integrate from $0$ to $t$ . The result is a new function $y_1$ . Repeat with $y_1$ to get $y_2$ , and so on. The dashed red curve is the true solution. Slide the iterate number up, and do it for all three equations.

For $y' = y$ and $y' = 2ty$ each new iterate agrees with the true solution to a higher power of $t$ , so the curves lock on from the left. For $y' = y^2$ they lock on only up to $t = 1$ . The true solution is $\frac{1}{1-t}$ , which rises to infinity at $t = 1$ , so no polynomial can follow it past that point. The theorem of this chapter says when the process converges and why the limit is the only solution. The example $y' = y^2$ shows what it cannot promise: a solution that lasts forever.

Where this came from· Picard, Lindelöf, Lipschitz, and an idea older than all of them

People solved differential equations by repeated substitution before anyone proved it works. Joseph Liouville used successive approximations in the 1830s to compute solutions. Rudolf Lipschitz proved existence and uniqueness in 1868, by Euler’s polygons, for right-hand sides satisfying the condition now named after him. Giuseppe Peano showed in 1886 that mere continuity of the right-hand side still gives a solution, though not a unique one. Émile Picard introduced successive approximations as a general existence method in a memoir of 1890 on partial differential equations, and in 1893 applied it to ordinary equations, which is the theorem of this chapter. Ernst Lindelöf presented a short, clean version for ordinary equations in 1894, which is why the theorem carries both names. Writing it as a contraction on a space of functions came later still: it is Banach’s fixed point theorem of 1922 applied to Picard’s operator.

The idea, in words

Why· A differential equation is a rule for motion; integrating turns the rule into a loop

The equation $y' = f(t, y)$ says: wherever you are, at time $t$ and position $y$ , your velocity is $f(t, y)$ . Starting from $y(t_0) = y_0$ , it is a rule you can follow, like a ball rolling on a landscape whose slopes are given. Two things can go wrong. Following the rule may lead you off in two different ways at once (no uniqueness), or may take you to infinity in finite time (no solution beyond that time). There is good news and a caveat. Neither failure happens if the rule does not change too abruptly from point to point (the Lipschitz condition). But even then the proof gives a solution only for a short time, and the solution may not be extendable forever.

The proof has three moves. Integrate the equation, so that the unknown function appears on both sides: $y(t) = y_0 + \int_{t_0}^{t} f(s, y(s))\,ds$ . Now a solution is a function that the “integrate the rule along me” machine $T$ leaves unchanged, that is, a fixed point of $T$ . Shrink: for a short enough time $T$ moves any two candidate functions closer together, by a factor below $1$ . That is exactly the Lipschitz condition at work inside an integral over a short interval. Use completeness: the continuous functions on that interval with the sup metric form a complete space (4.2), so Banach hands us exactly one fixed point.

Definitions and theorems

Differential equations and starting values

Definition T

Ordinary differential equation

An ordinary differential equation (ODE) is an equation for an unknown function $y$ of one real variable $t$ that involves $y$ and some of its derivatives. “Ordinary” means one variable (with several variables the derivatives are partial and the equation is a PDE). The order is the highest derivative that appears. In the explicit form of order $n$ ,

y^{(n)}(t) = f\big(t,\ y(t),\ y'(t),\ \dots,\ y^{(n-1)}(t)\big),

where $y^{(k)}$ is the $k$ -th derivative. A solution on an interval $J$ is a function $y$ with $n$ continuous derivatives on $J$ that satisfies the equation at every $t$ in $J$ . The values of $y$ may be numbers or, for a system, vectors in $\mathbb R^m$ . (Technau’s notes keep $J$ open and write $C^k(J)$ for the functions with $k$ continuous derivatives.)

In one lineAn equation for an unknown function $y(t)$ of ONE variable that involves its derivatives, such as $y' = 3y$ or $y'' + y = 0$ .

Why it existsIt is how a law of change (a growth rate, a force) is written down, with the unknown being the whole motion rather than a number.

The three equations of Technau’s lecture of March 20 show the range. $y' = ry$ is population growth (snakes, radioactive decay, interest on a bank balance). $y'' = -\omega y$ is a harmonic oscillator. $y' = ry(1 - y/K)$ is the logistic equation, in which growth slows as the population nears the capacity $K$ . The first two are linear (a sum of solutions is a solution, which 7.2 uses everywhere); the third is not.

Definition T

Initial value problem

An initial value problem (IVP) of first order is

y'(t) = f\big(t, y(t)\big), \qquad y(t_0) = y_0 .

A solution is a function $y$ on an interval $J \ni t_0$ that satisfies the equation at every point of $J$ and takes the value $y_0$ at $t_0$ . (For order $n$ one prescribes $y(t_0), y'(t_0), \dots, y^{(n-1)}(t_0)$ .)

In one lineAn ODE together with the value of the solution (and, for higher order, of its derivatives) at one starting time: $y' = f(t, y)$ , $y(t_0) = y_0$ .

Why it existsThe equation alone has a whole family of solutions. The starting value picks one, the way a starting position picks one path through a field of arrows.

Worked example MATH 376 worksheet 18 #5 to #8

Does the function solve the problem? y = e^(2x) for y' = 2y; y = x² + 1 for y' = 2x, y(0) = 1; y = x² for y' = x.

Solution.

Differentiate the candidate. $y = e^{2x}$ gives $y' = 2e^{2x}$ . To test a proposed solution we never solve anything. We differentiate it and substitute.
Substitute into the equation. $y' = 2e^{2x} = 2y$ . This is the equation $y' = 2y$ , so the first candidate works.
Second candidate: equation and starting value. $y = x^2 + 1$ has $y' = 2x$ , which matches. And $y(0) = 0 + 1 = 1$ . An IVP has two requirements, the equation and the starting value, and both must be checked.
Third candidate. $y = x^2$ gives $y' = 2x$ , but the equation asks for $y' = x$ , and $2x \ne x$ except at $x = 0$ . The equation must hold at every point of the interval, not at one point.

Answer. The first two are solutions (the second of the IVP). The third is not a solution.

Worked example MATH 376 worksheet 18 #9 to #12

Solve y' = e^t with y(1) = 0, y' = t² + 1 with y(0) = 3, y' = 3x with y(0) = 2, and y' = cos x with y(0) = 0.

Solution.

Integrate. When $f$ does not depend on $y$ , the equation $y' = f(t)$ asks for an antiderivative: $y = \int e^t\,dt = e^t + C$ . The right-hand side does not contain the unknown, so we are in the plain integration case of calculus.
Fix the constant with the starting value. $y(1) = e + C = 0$ gives $C = -e$ , so $y = e^t - e$ . The starting value is what selects one solution out of the family.
Second problem. $y = \frac{t^3}{3} + t + C$ and $y(0) = C = 3$ , so $y = \frac{t^3}{3} + t + 3$ . Same two moves: integrate, then fix $C$ .
Two more of the same kind. $y' = 3x$ with $y(0) = 2$ gives $y = \frac32x^2 + C$ and $C = 2$ . And $y' = \cos x$ with $y(0) = 0$ gives $y = \sin x + C$ and $C = 0$ . Same two moves again.
Check one of them. $\left(e^t - e\right)' = e^t$ and $e^1 - e = 0$ . Differentiating the answer and testing the starting value is the check from the last example.

Answer. $y = e^t - e$ , $y = \frac{t^3}{3} + t + 3$ , $y = \frac32x^2 + 2$ and $y = \sin x$ . These are the only cases where “solve the IVP” is only “integrate”. When $f$ contains $y$ , the unknown is inside the integral, and that is where the rest of the chapter begins.

Higher order is first order in disguise

Every explicit equation of order $n$ can be turned into a first-order equation for a vector. This is why we only need a theorem for first-order systems.

Fact T Technau, March 23 (Step 1 of Theorem 7.6); Apostol 7.1

A higher-order equation is a first-order system

Let $y$ be $n$ times differentiable and put $x_1 = y$ , $x_2 = y'$ , $\dots$ , $x_n = y^{(n-1)}$ . Then $y$ solves $y^{(n)} = f(t, y, y', \dots, y^{(n-1)})$ if and only if $x = (x_1, \dots, x_n)$ solves

x_1' = x_2,\quad x_2' = x_3,\quad \dots,\quad x_{n-1}' = x_n,\quad x_n' = f(t, x_1, x_2, \dots, x_n).

The starting values $y(t_0), \dots, y^{(n-1)}(t_0)$ become the starting vector $x(t_0)$ .

In one linePut $x = (y, y', \dots, y^{(n-1)})$ . Then $y^{(n)} = f(t, y, \dots, y^{(n-1)})$ is the same as $x' = F(t, x)$ for a vector function $F$ .

Why it is trueThe new unknown remembers the first $n - 1$ derivatives as separate coordinates, so each coordinate's derivative is either the next coordinate or the right-hand side.

ProofTechnau, March 23

The idea: Differentiate each coordinate and read off the pattern.

For

k < n

x_k' = y^{(k)} = x_{k+1}

The definition of

x_k

x_k = y^{(k-1)}

, so

x_k' = y^{(k)}

x_n' = y^{(n)} = f(t, y, \dots, y^{(n-1)}) = f(t, x_1, \dots, x_n)

The last coordinate is

x_n = y^{(n-1)}

, whose derivative is

y^{(n)}

, and the equation says what that is.

So solutions of the two problems correspond one to one.

Going the other way: if

x

solves the system, the first

n - 1

equations say that the coordinates are the successive derivatives of

x_1

, so

y = x_1

has

y^{(k)} = x_{k+1}

, and the last equation is the original one.

Worked example MATH 376 worksheet 28 #4; pendulum new

Write y'' + 4y = 0 as a first-order system. Do the same for the pendulum θ'' = −sin θ.

Solution.

Name the coordinates. For $y'' + 4y = 0$ put $x_1 = y$ and $x_2 = y'$ . An equation of order 2 needs two coordinates.
Differentiate each. $x_1' = y' = x_2$ and $x_2' = y'' = -4y = -4x_1$ . The first is the definition of $x_2$ . The second uses the equation to replace $y''$ .
Write it as a matrix equation. $\begin{bmatrix}x_1\\x_2\end{bmatrix}' = \begin{bmatrix}0 & 1\\-4 & 0\end{bmatrix}\begin{bmatrix}x_1\\x_2\end{bmatrix}$ . The equation is linear in $x$ , so the right-hand side is a matrix times $x$ . This matrix is called the companion matrix of the equation (7.2).
The pendulum. With $x_1 = \theta$ , $x_2 = \theta'$ : $x_1' = x_2$ and $x_2' = -\sin x_1$ . Here the right-hand side $F(x) = (x_2, -\sin x_1)$ is not a matrix times $x$ . The equation is not linear, but it is still a first-order system and the theorems below apply.

Answer. $x' = \begin{bmatrix}0 & 1\\-4 & 0\end{bmatrix}x$ and $x' = \begin{bmatrix}x_2\\-\sin x_1\end{bmatrix}$ .

The integral equation

Theorem T Technau, March 20 (Step 1); Apostol 7.21

An IVP is an integral equation

Let $f$ be continuous, and let $y$ be a continuous function on an interval $J \ni t_0$ whose graph stays where $f$ is defined. Then $y$ is a solution of the IVP on $J$ if and only if

y(t) = y_0 + \int_{t_0}^{t} f\big(s, y(s)\big)\,ds \qquad \text{for all } t \in J.

For a vector-valued $y$ the integral is taken coordinate by coordinate.

In one lineA continuous function $y$ solves $y' = f(t, y)$ , $y(t_0) = y_0$ on an interval exactly when $y(t) = y_0 + \int_{t_0}^{t} f(s, y(s))\,ds$ for every $t$ in it.

Why it is trueThe fundamental theorem of calculus runs in both directions: integrating the equation gives the integral form, and differentiating the integral form gives the equation back.

ProofTechnau, March 20; Apostol 7.21

The idea: Integrate the equation for one direction, differentiate the formula for the other.

(

\Rightarrow

)

y(t) - y(t_0) = \int_{t_0}^{t} y'(s)\,ds = \int_{t_0}^{t} f(s, y(s))\,ds

y

solves the IVP then

y'(s) = f(s, y(s))

is continuous in

s

(a composition of continuous functions), so the fundamental theorem of calculus, part 2 (1.5), applies to

y

y(t) = y_0 + \int_{t_0}^{t} f(s, y(s))\,ds

y(t_0) = y_0

is the starting value.

Differentiating the integral form gives

y'(t) = f(t, y(t))

(

\Leftarrow

) The integrand

s \mapsto f(s, y(s))

is continuous, because

y

and

f

are, so part 1 of the fundamental theorem of calculus says that its integral from

t_0

is differentiable in

t

with derivative

f(t, y(t))

And

y(t_0) = y_0 + 0 = y_0

\blacksquare

Put

t = t_0

: the integral from

t_0

t_0

0

The unknown now sits on both sides of the equation, inside the integral and outside it. That is what a fixed point looks like. Define a machine $T$ that takes a function $y$ and returns the function $(Ty)(t) = y_0 + \int_{t_0}^{t} f(s, y(s))\,ds$ . The theorem says: $y$ solves the IVP if and only if $Ty = y$ .

The Lipschitz condition

Definition T

Lipschitz condition

Let $D$ be a set of pairs $(t, x)$ with $x$ in $\mathbb R^m$ , and let $f : D \to \mathbb R^m$ . We say $f$ satisfies a Lipschitz condition (in $x$ , uniformly in $t$ ) on $D$ with Lipschitz constant $L \ge 0$ if

\lVert f(t, x) - f(t, z) \rVert \le L\,\lVert x - z \rVert \qquad \text{whenever } (t, x), (t, z) \in D .

Here $\lVert \cdot \rVert$ is the Euclidean length. The time $t$ is the same on both sides: the condition compares two positions at one time, not two times. It is the same inequality as in a contraction (4.2), with the one difference that $L$ is allowed to be large.

In one lineA function $f(t, x)$ satisfies a Lipschitz condition in $x$ with constant $L$ if $\lVert f(t, x) - f(t, z) \rVert \le L\,\lVert x - z \rVert$ for all $t$ and all $x, z$ .

Why it existsIt limits how fast the right-hand side can change when the position changes. That is the size of the contraction constant in the proof.

Worked example extends MATH 376 worksheet 19 #3

Which of these right-hand sides satisfy a Lipschitz condition in y, and with what L? f = 2y; f = sin y; f = y² on |y| at most 3; f = y² on all of R; f = |y|; f = 3y^(2/3) near y = 0.

Solution.

$f = 2y$ . $\lvert 2y - 2z \rvert = 2\lvert y - z \rvert$ , so $L = 2$ , and this holds for all $y, z$ . A linear function stretches every difference by one fixed factor.
$f = \sin y$ . By the mean value theorem $\lvert \sin y - \sin z \rvert = \lvert \cos c \rvert\,\lvert y - z \rvert \le \lvert y - z \rvert$ , so $L = 1$ on all of $\mathbb R$ . The slope of $f$ is at most $1$ in size, and the mean value theorem turns a slope bound into a Lipschitz bound.
$f = y^2$ on $\lvert y \rvert \le 3$ . $\lvert y^2 - z^2 \rvert = \lvert y + z \rvert\,\lvert y - z \rvert \le 6\,\lvert y - z \rvert$ , so $L = 6$ . Factor the difference of squares, and bound $\lvert y + z \rvert$ using the range.
$f = y^2$ on all of $\mathbb R$ . The factor $\lvert y + z \rvert$ is not bounded, so no single $L$ works: take $z = 0$ and $y$ large, then $\lvert f(y) - f(0) \rvert = y\,\lvert y - 0 \rvert$ and the ratio is $y$ . The condition fails globally. It holds on every bounded range, with a constant that grows with the range. This is why $y' = y^2$ will blow up.
$f = \lvert y \rvert$ . By the reverse triangle inequality $\big\lvert \lvert y \rvert - \lvert z \rvert \big\rvert \le \lvert y - z \rvert$ , so $L = 1$ . Lipschitz does not need a derivative. $\lvert y \rvert$ has a corner at $0$ and is still Lipschitz.
$f = 3y^{2/3}$ near $0$ . With $z = 0$ : $\lvert f(y) - f(0) \rvert = 3\lvert y \rvert^{2/3}$ , so the ratio to $\lvert y - 0 \rvert$ is $3\lvert y \rvert^{-1/3}$ , which tends to infinity as $y \to 0$ . The slope of $y^{2/3}$ is infinite at $0$ , and no $L$ can cover it.

Answer. $L = 2$ , $L = 1$ , $L = 6$ (on $\lvert y \rvert \le 3$ ), none on all of $\mathbb R$ , $L = 1$ , and none near $0$ for the last.

Theorem B standard (mean value theorem); Apostol 7.22

Derivative test for a Lipschitz condition

Let $f : D \to \mathbb R$ with $D = I \times K$ , $I$ an interval and $K$ an interval of positions, let $\partial f / \partial x$ exist on $D$ and satisfy $\lvert \partial f / \partial x \rvert \le L$ . Then $\lvert f(t, x) - f(t, z) \rvert \le L\,\lvert x - z \rvert$ for all $t \in I$ and $x, z \in K$ .

In one lineIf $f(t, x)$ is differentiable in the real variable $x$ on a region where $\lvert \partial f/\partial x \rvert \le L$ and the region contains the segment between any two positions at the same time, then $f$ satisfies a Lipschitz condition with constant $L$ .

Why it is trueFor two positions at the same time, the mean value theorem in $x$ says the change of $f$ is a slope times the change of $x$ , and the slope is at most $L$ .

Proofstandard

The idea: Fix the time and apply the mean value theorem to the function of position alone.

Let

x \ne z

K

. By the mean value theorem (1.4) there is

c

between

x

and

z

with

f(t, x) - f(t, z) = \frac{\partial f}{\partial x}(t, c)\,(x - z)

For fixed

t

, the function

x \mapsto f(t, x)

is differentiable on the interval

K

\lvert f(t, x) - f(t, z) \rvert \le L\,\lvert x - z \rvert

\blacksquare

c

is in

K

because

K

is an interval, so the bound on the derivative applies.

Quick checkFor which equation does the Lipschitz condition FAIL near the starting value

y(0) = 0

Picard iteration

Definition A

Picard iteration

For the IVP $y' = f(t, y)$ , $y(t_0) = x_0$ , the Picard operator is $(Ty)(t) = x_0 + \int_{t_0}^{t} f(s, y(s))\,ds$ , and Picard iteration produces the Picard iterates $y_0 = x_0$ , $y_1 = Ty_0$ , $y_2 = Ty_1$ , and so on. Apostol (7.21 to 7.23) calls this the method of successive approximations. (From here on the starting value is written $x_0$ , because the symbol $y_0$ now names the first iterate, the constant function $x_0$ .)

In one lineStart with the constant function $y_0(t) = x_0$ and repeatedly set $y_{k+1}(t) = x_0 + \int_{t_0}^{t} f(s, y_k(s))\,ds$ .

Why it existsIt is the contraction mapping iteration $y_{k+1} = Ty_k$ for the operator $T$ whose fixed points are the solutions. Each pass makes the guess agree with the solution to a higher order in $t - t_0$ .

Worked example compare Apostol 7.21

Picard iterates for y' = 2ty, y(0) = 1.

Solution. Use the stepper to watch the integral at each pass.

Answer. $y_n = \sum_{k=0}^{n} \frac{t^{2k}}{k!}$ , converging to $y = e^{t^2}$ for every $t$ .

Your answerIn the Picard iteration for

y' = 2ty

y(0) = 1

, what is

y_3(1)

? (Give a fraction.)

Worked example MATH 376 lecture of April 22 (y' = y²); y' = y as in 4.2

Picard iterates for y' = y, y(0) = 1, and for y' = y², y(0) = 1.

Solution.

$y' = y$ : first pass. $y_1 = 1 + \int_0^t 1\,ds = 1 + t$ . Here $f(s, y) = y$ and $y_0 = 1$ .
Continue. $y_2 = 1 + \int_0^t (1 + s)\,ds = 1 + t + \frac{t^2}{2}$ , and $y_n = \sum_{k=0}^{n} \frac{t^k}{k!}$ , which converges to $e^t$ . The partial sums of the exponential series of 1.6. This is the example computed in 4.2 with the contraction constant $a$ on $[0, a]$ .
$y' = y^2$ : first pass. $y_1 = 1 + \int_0^t 1\,ds = 1 + t$ . $f(s, y) = y^2$ , and $y_0^2 = 1$ .
Second and third pass. $y_2 = 1 + \int_0^t (1 + s)^2\,ds = 1 + t + t^2 + \frac{t^3}{3}$ , and $y_3 = 1 + t + t^2 + t^3 + \frac{2t^4}{3} + \frac{t^5}{3} + \frac{t^6}{9} + \frac{t^7}{63}$ . Square the previous polynomial and integrate. Each iterate agrees with $1 + t + t^2 + t^3 + \cdots$ for more terms: $y_1$ for 2, $y_2$ for 3, $y_3$ for 4.
Name the limit. The pattern $1 + t + t^2 + t^3 + \cdots$ is the geometric series $\frac{1}{1-t}$ , which converges only for $\lvert t \rvert < 1$ . One checks directly that $y = \frac{1}{1-t}$ satisfies $y' = \frac{1}{(1-t)^2} = y^2$ and $y(0) = 1$ .

Answer. $y' = y$ gives $e^t$ for all $t$ . $y' = y^2$ gives $\frac{1}{1-t}$ for $t < 1$ only: the solution blows up at $t = 1$ .

The Picard–Lindelöf theorem

Now we state and prove the theorem. It is Technau’s Theorem 7.4 of March 20, with one check that his notes skip (that the iterates stay inside the region where $f$ is defined) written in.

Theorem T Technau, March 20 (Theorem 7.4); Apostol 7.23, 7.28; Picard 1893, Lindelöf 1894

Picard–Lindelöf theorem

Let $R = \{(t, x) : \lvert t - t_0 \rvert \le a,\ \lVert x - x_0 \rVert \le b\}$ be a closed rectangle in $\mathbb R \times \mathbb R^m$ . Suppose $f : R \to \mathbb R^m$ is continuous and satisfies a Lipschitz condition in $x$ on $R$ with constant $L$ . Let $M = \max_R \lVert f \rVert$ and

h = \min\Big(a,\ \frac bM,\ \frac{1}{2L}\Big)

(read $\frac bM = \infty$ if $M = 0$ , and $\frac{1}{2L} = \infty$ if $L = 0$ ). Then the IVP $y' = f(t, y)$ , $y(t_0) = x_0$ has exactly one solution on $I = [t_0 - h, t_0 + h]$ (the same holds on any smaller interval around $t_0$ ). The Picard iterates converge uniformly on $I$ to it, with $\max_I \lVert y_n - y \rVert \le 2^{1-n}\,Mh$ .

In one lineIf $f$ is continuous and Lipschitz in $x$ on a rectangle around $(t_0, x_0)$ , the IVP has exactly one solution on a short interval around $t_0$ , and Picard iteration converges to it.

Why it is trueThe integral operator moves any two candidate functions at most $Lh$ times as far apart as before, where $h$ is half the length of the time interval. For $h$ small that is a contraction of a complete space.

Why· The picture behind the proof

In the picture of this chapter, the iterates $y_0, y_1, y_2, \dots$ are the green curves. The proof says that the vertical gap between consecutive iterates is at most half of the previous gap, at every $t$ in $[t_0 - h, t_0 + h]$ . A sequence of curves whose consecutive gaps shrink geometrically has summable gaps, so the curves bunch together in the tube sense of 4.2: Cauchy in the sup metric. Completeness of the space of continuous functions provides the limit curve, and the limit is the red dashed one. The three numbers in $h$ each protect one thing: $a$ keeps $t$ in the rectangle, $b / M$ keeps the curves inside it (a speed of at most $M$ for time $h$ moves a point at most $Mh \le b$ ), and $1/(2L)$ makes the gap shrink by $Lh \le \frac12$ per pass.

ProofTechnau, March 20; Apostol 7.28

The idea: Work in the space X of continuous curves on I that stay inside the rectangle. Show X is complete, that the Picard operator T sends X into X, and that T shrinks distances by Lh at most 1/2. Banach's theorem then gives exactly one fixed point, which by the integral equation theorem is the solution.

Let

(Ty)(t) = x_0 + \int_{t_0}^{t} f(s, y(s))\,ds

. We look for fixed points of

T

The integral equation theorem above: solutions of the IVP on

I

are exactly the continuous fixed points of

T

whose graph lies in

R

Let

X

be the set of continuous

y : I \to \mathbb R^m

with

\lVert y(t) - x_0 \rVert \le b

for all

t \in I

. It is non-empty (it contains the constant

x_0

) and complete.

C(I, \mathbb R^m)

with the sup metric

d(y, z) = \max_{t \in I} \lVert y(t) - z(t) \rVert

is complete: this is the theorem of 4.2, applied to each coordinate (convergence in

\mathbb R^m

is coordinatewise). Uniform limits keep an inequality

\lVert y(t) - x_0 \rVert \le b

, so

X

is a closed subset, hence complete by 4.2.This closed-ball restriction is the check that is missing in the lecture notes: without it

f(s, y(s))

might be asked for outside the rectangle, where

f

is not defined or not Lipschitz.

Ty

is a well-defined continuous function on

I

For

y \in X

the points

(s, y(s))

lie in

R

, so

f(s, y(s))

is defined and continuous in

s

(composition of continuous maps). An integral of a continuous function is continuous in its upper limit.

For

y \in X

and

t \in I

\lVert (Ty)(t) - x_0 \rVert = \big\lVert \int_{t_0}^{t} f(s, y(s))\,ds \big\rVert \le M\,\lvert t - t_0 \rvert \le Mh \le b

, since

h \le b/M

First the fact

\big\lVert \int v \big\rVert \le \int \lVert v \rVert

, proved after this proof. Then

\lVert f \rVert \le M

R

and

\lvert t - t_0 \rvert \le h

T

maps

X

into

X

This is why

h \le \frac bM

is in the definition of

h

: a curve with speed at most

M

cannot leave the tube of radius

b

in time

h

For

y, z \in X

\lVert (Ty)(t) - (Tz)(t) \rVert \le \Big\lvert \int_{t_0}^{t} \lVert f(s, y(s)) - f(s, z(s)) \rVert\,ds \Big\rvert \le L \,\Big\lvert \int_{t_0}^{t} \lVert y(s) - z(s) \rVert\,ds \Big\rvert \le L\,\lvert t - t_0 \rvert\,d(y, z)

The Lipschitz condition, applied at each time

s

to the two positions

y(s)

and

z(s)

, then

\lVert y(s) - z(s) \rVert \le d(y, z)

by the definition of the sup metric.

d(Ty, Tz) \le Lh\,d(y, z) \le \frac12\,d(y, z)

. So

T

is a contraction of

X

with

q = \frac12

Take the maximum over

t \in I

, using

\lvert t - t_0 \rvert \le h

and

h \le \frac{1}{2L}

.This is where the short time interval pays off: over a long interval the factor

L\lvert t - t_0 \rvert

could exceed 1.

T

has exactly one fixed point

y

X

. The Picard iterates

y_{k+1} = Ty_k

converge to it in the sup metric (uniformly), and

d(y_n, y) \le 2^{1-n}Mh

Banach fixed point theorem (4.2), parts (a), (b), (c). For (c):

d(y_1, y_0) = \max_t \lVert \int_{t_0}^{t} f(s, x_0)\,ds \rVert \le Mh

, and

\frac{q^n}{1-q} = 2 \cdot 2^{-n}

So the fixed point is a solution, and it is the only one.

\blacksquare

Any solution on

I

must have its graph in

R

, because

f

is only defined on

R

. So it is in

X

, and a fixed point of

T

by the integral equation theorem. The same argument works on a smaller interval around

t_0

(replace

I

by it).

The proof used one inequality for vector-valued integrals, which for numbers is the familiar $\lvert \int g \rvert \le \int \lvert g \rvert$ .

Fact B standard; uses Cauchy–Schwarz from 2.9

The norm of an integral

Let $v : [a, b] \to \mathbb R^m$ be continuous (the integral is coordinatewise). Then $\big\lVert \int_a^b v \big\rVert \le \int_a^b \lVert v \rVert$ .

In one lineFor a continuous curve $v : [a, b] \to \mathbb R^m$ , $\lVert \int_a^b v(s)\,ds \rVert \le \int_a^b \lVert v(s) \rVert\,ds$ .

Why it is truePair the integral with itself and use Cauchy–Schwarz inside the integral; then divide by its length.

Proofstandard

The idea: Let w be the integral. Compute w·w by moving the dot product inside the integral, bound the integrand with Cauchy–Schwarz, and divide by the length of w.

Let

w = \int_a^b v

. Then

\lVert w \rVert^2 = w \cdot \int_a^b v = \int_a^b w \cdot v(s)\,ds

Linearity of the integral: the dot product with a fixed vector can be moved inside.

\le \int_a^b \lVert w \rVert\,\lVert v(s) \rVert\,ds = \lVert w \rVert \int_a^b \lVert v \rVert

The Cauchy–Schwarz inequality of 2.9:

\lvert w \cdot v \rvert \le \lVert w \rVert\,\lVert v \rVert

. Integrals preserve inequalities.

\lVert w \rVert \le \int_a^b \lVert v \rVert

\blacksquare

w = O

there is nothing to prove; otherwise divide by

\lVert w \rVert > 0

Proof workshopRebuild the proof of the Picard–Lindelöf theoremTechnau, March 20

Tap the lines in the order a proof would write them. Each line can only use what is already above it.

Quick checkIn the proof above, which step uses completeness of the space of continuous functions?

Worked example MATH 376 worksheet 19 #3

Use the theorem for y' = 2y, y(0) = 1. How long does it promise a solution? Is there only one?

Solution.

Identify $f$ . $f(t, y) = 2y$ , continuous in both variables. Continuity of $f$ is the first hypothesis.
Check the Lipschitz condition. $\lvert 2y - 2z \rvert = 2\lvert y - z \rvert$ , so $L = 2$ on the whole plane. A function that is linear in $y$ is Lipschitz with $L$ the absolute value of its coefficient. This is why the worksheet says “linear in $y$ , so Lipschitz”.
Pick a rectangle and compute $h$ . Take $b = 1$ , $a = 1$ . On $\lvert y - 1 \rvert \le 1$ , $y \in [0, 2]$ and $M = \max \lvert 2y \rvert = 4$ . Then $h = \min(1, \frac14, \frac{1}{2 \cdot 2}) = \frac14$ . The theorem guarantees one solution on $[-\frac14, \frac14]$ , not more.
Name the solution. $y = e^{2t}$ : it satisfies $y' = 2e^{2t} = 2y$ and $y(0) = 1$ . Uniqueness says that this is THE solution. Any method that finds one solution has found them all.
Why the answer lasts longer than $h$ . The right-hand side is Lipschitz on the whole plane, so the global linear theorem below gives the solution on all of $\mathbb R$ . The number $h$ is a guarantee, not the true lifetime.

Answer. Exactly one solution exists on $[-\frac14, \frac14]$ by the theorem, namely $e^{2t}$ ; in fact it is the unique solution on all of $\mathbb R$ .

Uniqueness on a long interval, and linear equations

The theorem gives uniqueness only on a short interval $I$ . Two solutions that agree at $t_0$ must agree wherever both exist, not only on $I$ .

Corollary B standard; Apostol 7.23

Two solutions that meet are equal

Let $f$ be continuous on an open set $D$ of pairs $(t, x)$ and Lipschitz in $x$ on every closed rectangle in $D$ (a locally Lipschitz function; any $f$ with continuous $\partial f / \partial x$ is such). If $y$ and $z$ both solve the IVP $y' = f(t, y)$ , $y(t_0) = x_0$ on an interval $J \ni t_0$ , then $y = z$ on $J$ .

In one lineIf $f$ is continuous and satisfies a Lipschitz condition on every closed rectangle inside its domain, then two solutions of the same IVP agree on every interval where both are defined.

Why it is trueThe set where the two solutions agree is closed (by continuity) and open (by local uniqueness at each of its points). An interval cannot be split into two such sets unless one is empty.

Proofstandard

The idea: Let S be the set of times where the two solutions are equal. It contains t0. Show S is closed and open in J; a connected interval has no other such subsets.

Let

S = \{t \in J : y(t) = z(t)\}

. Then

t_0 \in S

and

S

is closed in

J

y

and

z

are continuous, so

y - z

is continuous and the set where it is zero is closed in

J

s \in S

, then

y

and

z

are two solutions of the IVP with starting point

(s, y(s))

that stay in

R

near

s

, so they agree on an interval around

s

Take

s \in S

and a closed rectangle

R

around

(s, y(s))

inside

D

. By continuity both

y(t)

and

z(t)

stay in

R

for

t

near

s

. Apply the uniqueness part of Picard–Lindelöf with starting time

s

Hence

S

is open in

J

S

contains a neighbourhood of each of its points: it is open in

J

S = J

\blacksquare

An interval is connected: a non-empty subset that is both open and closed in it is all of it.

For linear systems the right-hand side is Lipschitz on the whole space, and then the solution lasts as long as the coefficients do.

Theorem B Technau, March 23 ('one can show'); Apostol 7.18 and 7.21; Rudin 9.23 idea

Linear systems have solutions on the whole interval

Let $J$ be an interval, $A : J \to \mathbb R^{m \times m}$ and $g : J \to \mathbb R^m$ continuous, $t_0 \in J$ , $x_0 \in \mathbb R^m$ . Then the IVP $x' = A(t)x + g(t)$ , $x(t_0) = x_0$ has exactly one solution on $J$ .

In one lineIf $A(t)$ and $g(t)$ are continuous on an interval $J$ , then $x' = A(t)x + g(t)$ , $x(t_0) = x_0$ has exactly one solution, and it is defined on all of $J$ .

Why it is trueOn a closed bounded interval the Picard operator is not a contraction, but its $m$ -th power is, because the shrinking factor becomes $(L\ell)^m / m!$ , which falls below 1 for large $m$ .

ProofTechnau, March 23; 4.2 stretch problem

The idea: It is enough to work on a closed bounded interval K inside J. The Picard operator T has Lipschitz constant L for the right-hand side on all of R^m, so no restriction on the size of the functions is needed. T itself may not be a contraction on K, but T^m is: induct to get a factor (L times the length)^m divided by m!, then use that some power of T is a contraction.

Fix a closed bounded interval

K = [\alpha, \beta] \subseteq J

containing

t_0

, and write

\ell = \beta - \alpha

. We find a unique solution on

K

Any two points of

J

lie in a closed bounded subinterval, and solutions on bigger intervals restrict to smaller ones. By the corollary, solutions on different

K

agree where they overlap and fit together into one solution on

J

Let

L = \max_{t \in K} \lVert A(t) \rVert

. Then

\lVert A(t)u - A(t)w \rVert \le L\,\lVert u - w \rVert

for all vectors

u, w

and

t \in K

The operator norm of 3.5:

\lVert A(t)u \rVert \le \lVert A(t) \rVert\,\lVert u \rVert

. The continuous function

t \mapsto \lVert A(t) \rVert

has a maximum on

K

Let

(Ty)(t) = x_0 + \int_{t_0}^{t} \big(A(s)y(s) + g(s)\big)\,ds

X = C(K, \mathbb R^m)

with the sup metric

d

X = C(K, \mathbb R^m)

is complete (4.2). There is no ball

b

now: the right-hand side is defined for every

x

Claim:

\lVert (T^my)(t) - (T^mz)(t) \rVert \le \dfrac{(L\lvert t - t_0 \rvert)^m}{m!}\,d(y, z)

for all

y, z \in X

t \in K

m \ge 0

Induction on

m

. Case

m = 0

: both sides are

\lVert y - z \rVert \le d(y, z)

. Inductive step: apply the Lipschitz estimate under the integral and integrate

\lvert s - t_0 \rvert^m

, which gives

\lvert t - t_0 \rvert^{m+1} / (m+1)

.This is the same computation as the one-step estimate

L\lvert t - t_0 \rvert\,d(y, z)

, with the previous bound plugged in instead of the crude

d(y, z)

Hence

d(T^my, T^mz) \le \frac{(L\ell)^m}{m!}\,d(y, z)

, and for large

m

the factor is below

1

S = T^m

is a contraction of

X

(L\ell)^m / m!

is the general term of the series for

e^{L\ell}

, so it tends to

0

(1.6). Choose

m

with

(L\ell)^m / m! < 1

T

itself has exactly one fixed point: the IVP has exactly one solution on

K

\blacksquare

Banach gives a unique fixed point

y^*

S

. Then

S(Ty^*) = T(Sy^*) = Ty^*

, so

Ty^*

is also a fixed point of

S

, hence equals

y^*

by uniqueness. A fixed point of

T

is a fixed point of

S

, so there is only one.

Proof workshopRebuild: a linear system has a solution on the whole intervalTechnau, March 23

Each line of the proof is shown in order. Choose the reason that justifies it, then press Check.

Fix a closed bounded interval $K$ of length $\ell$ . Let $L = \max_K \lVert A(t) \rVert$ .
$\lVert (T^my)(t) - (T^mz)(t) \rVert \le \dfrac{(L\lvert t - t_0 \rvert)^m}{m!}\,d(y, z)$ .
For large $m$ , $\dfrac{(L\ell)^m}{m!} < 1$ , so $T^m$ is a contraction.
$T^m$ has a unique fixed point $y^*$ .
$Ty^*$ is also a fixed point of $T^m$ , so $Ty^* = y^*$ .

Worked example

Why does the IVP y' = 2y, y(0) = 1 have a solution on all of R, even though the theorem only guarantees [−1/4, 1/4]?

Solution.

Write it as a linear system. $y' = A(t)y + g(t)$ with $A(t) = 2$ (a $1 \times 1$ matrix) and $g = 0$ . The right-hand side is linear in $y$ , so the global linear theorem applies.
Check continuity. $A$ and $g$ are constants, so continuous on all of $\mathbb R$ . That is the only hypothesis.
Apply the theorem to $K = [-N, N]$ for each $N$ . There is a unique solution on every $[-N, N]$ ; by uniqueness the solutions fit together. The short $h$ of the local theorem comes from the ball $b$ , which is not needed for linear equations.

Answer. The unique solution $y = e^{2t}$ is defined on all of $\mathbb R$ . For a non-linear $f$ the next example shows this can fail.

When the hypotheses fail

Worked example Apostol 6.1 style; new numbers

Non-uniqueness: show that y' = 3y^(2/3), y(0) = 0 has more than one solution.

Solution.

Find one solution by inspection. $y = 0$ has $y' = 0 = 3 \cdot 0^{2/3}$ and $y(0) = 0$ . The constant function at the starting value solves the problem.
Find another. Try $y = t^3$ . Then $y' = 3t^2$ and $3y^{2/3} = 3(t^3)^{2/3} = 3t^2$ . And $y(0) = 0$ . For $t \ge 0$ , $(t^3)^{2/3} = t^2$ . The cube root of a negative number is real, and $(t^3)^{2/3} = t^2$ for $t < 0$ as well.
Find a whole family. For any $c \ge 0$ , let $y_c(t) = 0$ for $t \le c$ and $y_c(t) = (t - c)^3$ for $t \ge c$ . Each is differentiable (the derivative $3(t - c)^2$ tends to $0$ at $c$ ) and satisfies the equation. The solution sits at $0$ for as long as it likes, and then leaves along a cubic. A solution may leave $0$ at ANY time.
See which hypothesis fails. $f(y) = 3y^{2/3}$ is continuous but not Lipschitz near $0$ (the earlier example). With a Lipschitz $f$ the Picard operator would be a contraction, and there would be one solution. Existence still holds (this is Peano’s theorem); uniqueness is what is lost.

Answer. There are infinitely many solutions, one for each leaving time $c \in [0, \infty]$ .

Worked example MATH 376 lecture of March 23 (remarks after Theorem 7.6)

What each hypothesis does: t y' = 2y with y(0) = 0; y'' = 0 with too few or too many conditions; why the logistic equation is not linear.

Solution.

Continuous coefficients. Write $ty' = 2y$ as $y' - \frac2t\,y = 0$ . The coefficient $-\frac2t$ is not defined at $t = 0$ , so the linear theorem of 7.2 does not apply on $\mathbb R$ . The theorem asks for continuous coefficients on the whole interval.
Two solutions with the same start. $y = t^2$ has $ty' = 2t^2 = 2y$ . And $y = t\lvert t\rvert$ has $y' = 2\lvert t\rvert$ and $ty' = 2t\lvert t\rvert = 2y$ . Both have $y(0) = 0$ and they are different functions. Uniqueness fails exactly where the coefficient breaks down. (These two functions return in 7.3 as independent functions with Wronskian $0$ .)
Too few conditions. $y'' = 0$ with only $y(0) = 0$ is solved by every $y = ct$ . An equation of order $n$ needs $n$ starting numbers; with fewer, uniqueness fails.
Too many conditions. $y'' = 0$ with $y(0) = 0$ , $y'(0) = 0$ and also $y(1) = 1$ has no solution, because the first two conditions force $y = 0$ . With more than $n$ conditions, existence can fail.
Linearity. In $y' = ry(1 - y/K)$ the constant $y = K$ is a solution, but $y = 2K$ is not: $y' = 0$ while $r \cdot 2K(1 - 2) = -2rK \ne 0$ . A solution doubled must again be a solution if the equation is linear. Here it is not, because of the term $-ry^2/K$ .

Answer. The theorem needs continuous coefficients, exactly $n$ conditions, and (for the whole interval) linearity. Each example above breaks one of them.

Worked example MATH 376 lecture of April 22 (Example 8.14)

Blow-up: solve y' = y², y(0) = 1. Why does the solution stop at t = 1, and what does the theorem promise?

Solution.

Separate variables. For $y \ne 0$ : $\frac{y'}{y^2} = 1$ , so $\frac{d}{dt}\left(-\frac1y\right) = 1$ , so $-\frac1y = t + C$ . The left side is the derivative of $-1/y$ .
Fix the constant. $y(0) = 1$ gives $C = -1$ , so $y = \frac{1}{1 - t}$ . Check: $y' = \frac{1}{(1-t)^2} = y^2$ and $y(0) = 1$ .
Read off the lifetime. The formula is a solution on $(-\infty, 1)$ and tends to $+\infty$ as $t \to 1$ from below. It cannot be continued to $t = 1$ . A solution of an ODE is a function on an interval with finite values. After $t = 1$ nothing is left.
Compute what the theorem promises. On the rectangle $\lvert y - 1 \rvert \le b$ we have $M = (1 + b)^2$ and $L = 2(1 + b)$ , so $h = \min\big(\frac{b}{(1+b)^2}, \frac{1}{4(1+b)}\big)$ . The two terms are equal when $b = \frac13$ , and then $h = \frac{3}{16} = 0.1875$ . This is the best the theorem can do for this rectangle family. It is much less than the true $1$ , as the theorem only uses worst-case information.
Why this does not contradict the linear theorem. The right-hand side $y^2$ is not Lipschitz on all of $\mathbb R$ (shown earlier). The larger $y$ is, the faster it grows, and the faster $y$ grows, the larger $y$ gets. That feedback loop reaches infinity in finite time.
General starting value. With $y(0) = y_0 > 0$ the solution is $y = \frac{y_0}{1 - y_0 t}$ , which blows up at $t = \frac{1}{y_0}$ . Technau’s power-series solution of April 22 finds the same formula: with $y = \sum \alpha_k t^k$ the recurrence gives $\alpha_k = \alpha_0^{k+1}$ , a geometric series with radius $1/\alpha_0$ .

Answer. $y = \frac{1}{1-t}$ on $(-\infty, 1)$ ; blow-up at $t = 1$ . The theorem guarantees only $[-\frac{3}{16}, \frac{3}{16}]$ for this rectangle.

Your answerThe solution of

y' = y^2

with

y(0) = 2

becomes infinite at a finite time

T

. What is

T

Your answerFor

f(y) = y^2

on the range

\lvert y \rvert \le 3

, what is the smallest Lipschitz constant

L

Beyond the course: for understanding· B: Peano's theorem, Euler polygons, and what the theorem leaves out

If $f$ is merely continuous, a solution still exists, locally (Peano, 1886; he extended it to systems in 1890), but it need not be unique, as $y' = 3y^{2/3}$ shows. A different construction proves it: Euler’s method with a smaller and smaller step $h$ gives polygons, and a subsequence of them converges to a solution (the Arzelà–Ascoli theorem, from MATH 521). With the Lipschitz condition the whole sequence converges and the limit is unique, which is Lipschitz’s original proof. The next lab runs Euler’s method and shows its error.

Python labPicard iterates as exact polynomials

import numpy as np
from numpy.polynomial import Polynomial as P

# Picard iteration for y' = f(t, y), y(0) = 1, with the iterates kept as exact polynomials
def picard(step, n):
    y, out = P([1.0]), [P([1.0])]
    for _ in range(n):
        y = step(y)
        out.append(y)
    return out

step = lambda y: 1 + (P([0, 2]) * y).integ()      # y' = 2 t y   (exact solution e^(t^2))
exact = lambda t: np.exp(t**2)

ts = np.linspace(0, 1, 401)
print(" n   largest gap on [0, 1]")
for n, y in enumerate(picard(step, 8)):
    print(f"{n:2d}   {np.max(np.abs(exact(ts) - y(ts))):.2e}")

Try this. For $y' = 2ty$ the largest gap on $[0, 1]$ falls from $1.7$ at $n = 0$ to $3 \times 10^{-6}$ at $n = 8$ , shrinking faster at each pass. Now replace the step with step = lambda y: 1 + (y * y).integ() (that is $y' = y^2$ ), the exact function with lambda t: 1 / (1 - t), change the $8$ in the loop to $10$ , and use np.linspace(0, 0.5, 401) instead of np.linspace(0, 1, 401): the gap falls steadily and is below $10^{-6}$ by $n = 10$ (it is $4.5 \times 10^{-7}$ ). With np.linspace(0, 0.9, 401) it is still about $0.18$ after 10 passes. The iterates are working against the blow-up at $t = 1$ .

Python labEuler's method: walk along the tangent, then re-aim

import math

# Euler's method for y' = f(t, y), y(0) = 1: walk along the tangent for a short time h, then re-aim.
f = lambda t, y: y             # y' = y
exact = math.exp               # e^t
T = 1.0

def euler(f, T, h):
    t, y = 0.0, 1.0
    for _ in range(round(T / h)):
        y += h * f(t, y)
        t += h
    return y

print("   h      Euler y(1)   error      error / h")
for h in [0.5, 0.1, 0.05, 0.025, 0.0125]:
    y = euler(f, T, h)
    print(f"{h:7.4f}  {y:10.6f}  {abs(exact(T) - y):.2e}  {abs(exact(T) - y) / h:.3f}")

Try this. Run it: halving $h$ roughly halves the error, so the last column settles near $e/2 = 1.36$ . Euler's method is first order. Now try a decaying equation: set f = lambda t, y: -5 * y, exact = lambda t: math.exp(-5 * t), T = 2.0 and the list [0.5, 0.4, 0.2, 0.1]. The true value is $e^{-10} = 4.5 \times 10^{-5}$ . With $h = 0.5$ Euler returns $5.06$ , with $h = 0.4$ it returns $-1$ , with $h = 0.2$ it returns $0$ , and only $h = 0.1$ gets close (about $10^{-6}$ ). A step that is too long amplifies the solution instead of shrinking it.

Worked example

Euler's method by hand: y' = t − y, y(0) = 1, with step h = 0.5 up to t = 1.

Solution.

State the rule. From $(t_k, y_k)$ move to $t_{k+1} = t_k + h$ and $y_{k+1} = y_k + h\,f(t_k, y_k)$ . Follow the tangent line for time $h$ , then recompute the slope at the new point.
First step. $f(0, 1) = 0 - 1 = -1$ , so $y_1 = 1 + 0.5(-1) = 0.5$ at $t = 0.5$ . The slope at the start is $-1$ .
Second step. $f(0.5, 0.5) = 0.5 - 0.5 = 0$ , so $y_2 = 0.5 + 0.5 \cdot 0 = 0.5$ at $t = 1$ . The slope at the new point is $0$ .
Compare with the exact solution. Solving the linear equation gives $y = t - 1 + 2e^{-t}$ , so $y(1) = 2e^{-1} = 0.7358$ . The error $0.236$ is large because $h = 0.5$ is large. With the next lab you can see it fall in proportion to $h$ .

Answer. $y_2 = 0.5$ , against the exact value $0.7358$ .

Your answerEuler's method with

h = 0.5

for

y' = y

y(0) = 1

: what is the approximation of

y(1)

Where it is used

Used inside maths· Inside maths: why the solution spaces of 7.2 have dimension n, and why orbits never cross

The linear theorem above is the engine of 7.2. It says: for each choice of $n$ starting values there is exactly one solution of a linear equation of order $n$ , on the whole interval. In the language of 2.4, the map “solution $\mapsto$ its starting values” is one-to-one and onto, an isomorphism, so the space of solutions has dimension $n$ . A second consequence is that two different solutions of $x' = F(x)$ never meet: if they met at time $s$ they would be two solutions of the same IVP starting at $s$ , and uniqueness says they are one. This is why the arrows of a phase portrait in 7.4 can bend but never cross. The same contraction argument also proves the inverse function theorem and the implicit function theorem (Unit 5).

Used in the real world· Biology: a bacterial culture with limited food (logistic growth)

Let $N(t)$ be the number of cells. With plenty of food, $N' = rN$ and $N = N_0e^{rt}$ . With limited food, growth slows near the capacity $K$ : $N' = rN(1 - N/K)$ . Take $r = 0.7$ per hour, $K = 10^9$ cells and $N_0 = 10^6$ . The right-hand side $f(N) = rN(1 - N/K)$ has $f'(N) = r(1 - 2N/K)$ , of size at most $3r = 2.1$ on $0 \le N \le 2K$ , so it is Lipschitz there and the theorem applies: there is exactly one solution for each $N_0$ . It is

N(t) = \frac{K}{1 + \left(\frac{K}{N_0} - 1\right)e^{-rt}},

(substitute to check). The culture reaches $K/2$ when $\left(\frac{K}{N_0} - 1\right)e^{-rt} = 1$ , that is $t = \frac{\ln 999}{0.7} \approx 9.87$ hours. At first the doubling time is $\frac{\ln 2}{0.7} \approx 0.99$ hours, as with $N' = rN$ . At $24$ hours the logistic model gives $N \approx 0.99995 \times 10^9$ , while the pure exponential would have predicted $1.98 \times 10^{13}$ cells. Technau’s lecture of March 20 introduced exactly this equation, with snakes, as the “more realistic” model after $y' = ry$ .

Used in the real world· Machine learning: residual networks and neural ODEs

A residual block computes $x_{k+1} = x_k + f(x_k)$ , where $f$ is a small neural network. Compare Euler’s method with step $h$ : $x_{k+1} = x_k + h\,f(x_k)$ . A stack of blocks is Euler’s method for the equation $x' = f(x)$ , with the layer number as time. In 2018 Chen, Rubanova, Bettencourt and Duvenaud posted the paper “Neural Ordinary Differential Equations”, which makes this literal: the network is the ODE, and a numerical solver decides how many steps to take. The theorem of this chapter is why the idea is well defined: if $f$ is Lipschitz (a network with weights of bounded size, and smooth activation functions, is) then the equation has a unique solution from each input $x(0)$ , so the map “input $\mapsto$ output at time $T$ ” is a well-defined function, and, running the equation backwards, one-to-one. The Lipschitz constant $L$ controls how far two nearby inputs can move apart: by Gronwall’s inequality, at most by the factor $e^{LT}$ after time $T$ . This is the reason practitioners penalise the size of the weights.

Traps

Watch out· Check the starting value as well as the equation

A function can satisfy $y' = 2x$ and have the wrong starting value: $y = x^2 + 5$ solves the equation and fails $y(0) = 1$ . The family $y = x^2 + C$ solves the equation for every $C$ , and the starting value picks one. Worksheet 18 #6 is built around this: verify both.

Watch out· Uniqueness needs the Lipschitz condition, and Lipschitz means the same time on both sides

$y' = 3y^{2/3}$ , $y(0) = 0$ has infinitely many solutions. The Lipschitz inequality compares $f(t, x)$ and $f(t, z)$ at the SAME $t$ . It does not say $f$ is continuous in $t$ (that is a separate hypothesis) and it is not a bound on $\lvert f(t, x) - f(s, x) \rvert$ .

Watch out· 'There is a solution' is local; it may end

$y' = y^2$ , $y(0) = 1$ has the solution $\frac{1}{1-t}$ , which does not exist at $t = 1$ , although the equation looks harmless. Only for linear equations (Lipschitz on the whole space) does the solution last as long as the coefficients are defined. Do not write “for all $t \in \mathbb R$ ” unless you have checked.

Watch out· Contraction needs q below 1 strictly, and the time interval is chosen to make it so

The estimate gives $d(Ty, Tz) \le L\lvert t - t_0 \rvert\,d(y, z)$ . For $L \lvert t - t_0 \rvert \ge 1$ the operator may not shrink anything, so the proof restricts to $\lvert t - t_0 \rvert \le h$ with $Lh \le \frac12$ . This is the same point as worksheet 18 #3: $T(x) = x + 1$ moves every distance by factor exactly $1$ and has no fixed point. In Technau’s lecture, $\varepsilon < \frac1L$ is the same condition.

Watch out· The iterates must stay inside the region where f is defined

The lecture proof applies the Lipschitz condition to $f(\tau, g(\tau))$ without checking that $g(\tau)$ lies in the rectangle where the condition holds. The fix is the ball of radius $b$ and the condition $h \le b/M$ . If you write this proof on an exam, say in one line why $T$ maps the space into itself. It is the same trap as in 4.2: always check $T(X) \subseteq X$ .

Watch out· Cauchy is not convergent, and convergent implies Cauchy always

On the 376 final (4(a)) you defined a Cauchy sequence as one whose norms converge, and on 6(c) you said that a convergent sequence in an incomplete space need not be Cauchy. A sequence is Cauchy if its terms get close to each other: for every $\varepsilon$ there is $N$ with $\lVert x_m - x_n \rVert < \varepsilon$ for $m, n \ge N$ . Every convergent sequence is Cauchy (triangle inequality through the limit, no completeness used). The converse is what completeness means, and the Picard–Lindelöf proof is where you use it.

Problems

checkMATH 376 worksheet 18 #5 to #8

Which of these solve the stated problem? (a) $y = e^{2x}$ for $y' = 2y$ . (b) $y = x^2 + 1$ for $y' = 2x$ , $y(0) = 1$ . (c) $y = \sin x$ for $y' = \cos x$ , $y(0) = 0$ . (d) $y = x^2$ for $y' = x$ . (e) $y = x^2 + 5$ for $y' = 2x$ , $y(0) = 1$ .

Solution

(a) Yes: $y' = 2e^{2x} = 2y$ . (b) Yes: $y' = 2x$ and $y(0) = 1$ . (c) Yes: $y' = \cos x$ and $\sin 0 = 0$ . (d) No: $y' = 2x \ne x$ . (e) No: the equation holds but $y(0) = 5 \ne 1$ .

check

For each right-hand side, state whether it satisfies a Lipschitz condition in $y$ on all of $\mathbb R$ , and give $L$ : (a) $f = -3y + t$ ; (b) $f = \cos y$ ; (c) $f = y^3$ ; (d) $f = \sqrt{\lvert y \rvert}$ .

Solution

(a) Yes, $L = 3$ : $\lvert (-3y + t) - (-3z + t) \rvert = 3\lvert y - z \rvert$ . (b) Yes, $L = 1$ since $\lvert \sin \rvert \le 1$ . (c) No: $\lvert y^3 - z^3 \rvert = \lvert y - z \rvert (y^2 + yz + z^2)$ and the second factor is unbounded; it is Lipschitz on bounded ranges, with $L = 3K^2$ on $\lvert y \rvert \le K$ . (d) No: near $0$ the ratio $\frac{\sqrt{\lvert y \rvert}}{\lvert y \rvert} = \lvert y \rvert^{-1/2}$ is unbounded. (This is a second example of non-uniqueness: $y' = \sqrt{\lvert y \rvert}$ , $y(0) = 0$ has the solutions $0$ and $t^2/4$ for $t \ge 0$ .)

checkMATH 376 worksheet 18 #3 and #4

Is $T(x) = x + 1$ a contraction of $\mathbb R$ ? Does it have a fixed point? Is $T(x) = \cos x$ a contraction of $[0, 1]$ ?

Solution

$\lvert T(x) - T(y) \rvert = \lvert x - y \rvert$ , so the best constant is $q = 1$ , not below $1$ : not a contraction. $x + 1 = x$ is impossible, so there is no fixed point, which shows the hypothesis $q < 1$ cannot be dropped. For $\cos$ on $[0, 1]$ : it maps $[0, 1]$ into $[\cos 1, 1] \subseteq [0, 1]$ , and $\lvert \cos' \rvert = \lvert \sin x \rvert \le \sin 1 < 1$ , so yes, with $q = \sin 1$ .

checkMATH 376 practice final 4(a)

State Banach’s fixed point theorem. (Write all hypotheses.)

Solution

Let $(X, d)$ be a non-empty complete metric space (for example a Banach space, which is a complete normed space) and $T : X \to X$ a contraction: there is $q$ with $0 \le q < 1$ and $d(Tx, Ty) \le q\,d(x, y)$ for all $x, y$ . Then $T$ has exactly one fixed point $x^*$ ( $Tx^* = x^*$ ), and for every starting point $x_0$ the iterates $x_{n+1} = Tx_n$ converge to $x^*$ . (Optional: $d(x_n, x^*) \le \frac{q^n}{1 - q}\,d(x_1, x_0)$ .)

use

Do three Picard iterations for $y' = y + 1$ , $y(0) = 0$ . What function do the iterates approach, and what is it?

Solution

$y_0 = 0$ . $y_1 = \int_0^t (0 + 1)\,ds = t$ . $y_2 = \int_0^t (s + 1)\,ds = t + \frac{t^2}{2}$ . $y_3 = \int_0^t \left(s + \frac{s^2}{2} + 1\right)ds = t + \frac{t^2}{2} + \frac{t^3}{6}$ . In general $y_n = \sum_{k=1}^{n} \frac{t^k}{k!}$ , which tends to $e^t - 1$ . Check: $y = e^t - 1$ has $y' = e^t = y + 1$ and $y(0) = 0$ .

use

For $y' = t - y$ , $y(0) = 1$ , find $y_1, y_2, y_3$ by Picard iteration, and compare with the exact solution $y = t - 1 + 2e^{-t}$ at $t = 0.5$ .

Solution

$y_0 = 1$ . $y_1 = 1 + \int_0^t (s - 1)\,ds = 1 - t + \frac{t^2}{2}$ . $y_2 = 1 + \int_0^t \left(s - 1 + s - \frac{s^2}{2}\right)ds = 1 - t + t^2 - \frac{t^3}{6}$ . $y_3 = 1 + \int_0^t \left(s - 1 + s - s^2 + \frac{s^3}{6}\right)ds = 1 - t + t^2 - \frac{t^3}{3} + \frac{t^4}{24}$ . At $t = 0.5$ : $y_3 = 1 - 0.5 + 0.25 - 0.041667 + 0.002604 = 0.710938$ . Exact: $0.5 - 1 + 2e^{-0.5} = 0.713061$ . The gap is $0.0021$ .

use

For $y' = y^2$ with $y(0) = 2$ , find the guaranteed $h$ for the rectangle $\lvert y - 2 \rvert \le 1$ with $a = 1$ , and compare with the true blow-up time.

Solution

On $y \in [1, 3]$ : $M = 9$ , $L = \max \lvert 2y \rvert = 6$ . So $h = \min(1, \frac19, \frac{1}{12}) = \frac1{12} \approx 0.083$ . The true solution is $\frac{2}{1 - 2t}$ , which blows up at $t = \frac12$ . The theorem guarantees a solution on $[-\frac{1}{12}, \frac{1}{12}]$ , about one sixth of the true lifetime.

use

Write $y''' + 2y'' - y' + 3y = \sin t$ as a first-order system $x' = Ax + g(t)$ , and say which theorem of this chapter proves that its IVP has exactly one solution on $\mathbb R$ .

Solution

With $x = (y, y', y'')$ : $x_1' = x_2$ , $x_2' = x_3$ , $x_3' = y''' = -3x_1 + x_2 - 2x_3 + \sin t$ . So $A = \begin{bmatrix}0 & 1 & 0\\0 & 0 & 1\\-3 & 1 & -2\end{bmatrix}$ and $g = (0, 0, \sin t)$ . $A$ is constant and $g$ is continuous on $\mathbb R$ , so the global linear theorem gives exactly one solution on $\mathbb R$ for each $x(0) = (y(0), y'(0), y''(0))$ .

prove

Prove that if $f$ is continuous and $y$ is continuous, then $y$ solves $y' = f(t, y)$ , $y(t_0) = y_0$ on $J$ if and only if $y(t) = y_0 + \int_{t_0}^{t} f(s, y(s))\,ds$ on $J$ . (This is the first theorem of the chapter, rewritten as an exercise.)

Solution

Plan. Use the fundamental theorem of calculus in each direction.

( $\Rightarrow$ ) If $y' = f(t, y)$ then $y'$ is continuous, since $f$ and $y$ are. By part 2 of the fundamental theorem, $y(t) - y(t_0) = \int_{t_0}^{t} y'(s)\,ds = \int_{t_0}^{t} f(s, y(s))\,ds$ , and $y(t_0) = y_0$ . ( $\Leftarrow$ ) The function $s \mapsto f(s, y(s))$ is continuous, so by part 1 the right-hand side is differentiable with derivative $f(t, y(t))$ . So $y' = f(t, y)$ . Put $t = t_0$ : the integral is $0$ , so $y(t_0) = y_0$ . $\blacksquare$

proveApostol 7.22 idea

Prove by induction that for $y' = y$ , $y(0) = 1$ , the Picard iterates satisfy $\lvert y_{k+1}(t) - y_k(t) \rvert \le \frac{\lvert t \rvert^{k+1}}{(k+1)!}$ . Deduce that they converge uniformly on every $[-N, N]$ , without using the factor $Lh \le \frac12$ .

Solution

Plan. Subtract consecutive iterates, integrate, and use the series for $e^N$ .

$y_1 - y_0 = t$ , so $\lvert y_1 - y_0 \rvert = \lvert t \rvert = \frac{\lvert t \rvert^1}{1!}$ . If $\lvert y_{k+1} - y_k \rvert \le \frac{\lvert s \rvert^{k+1}}{(k+1)!}$ then $\lvert y_{k+2}(t) - y_{k+1}(t) \rvert = \left\lvert \int_0^t (y_{k+1} - y_k)\,ds \right\rvert \le \left\lvert \int_0^t \frac{\lvert s \rvert^{k+1}}{(k+1)!}\,ds \right\rvert = \frac{\lvert t \rvert^{k+2}}{(k+2)!}$ . On $[-N, N]$ the gaps are at most $\frac{N^{k+1}}{(k+1)!}$ , and $\sum_k \frac{N^{k+1}}{(k+1)!} < \infty$ (it is the exponential series). By the Weierstrass M-test (1.6) the series $y_0 + \sum_k (y_{k+1} - y_k)$ converges uniformly, and its partial sums are the $y_k$ . $\blacksquare$

prove

Let $x$ and $z$ be two solutions of $x' = f(t, x)$ on $[t_0, t_1]$ , where $f$ is Lipschitz in $x$ with constant $L$ on a set containing both graphs. Prove that $\lVert x(t) - z(t) \rVert \le \lVert x(t_0) - z(t_0) \rVert\,e^{L(t - t_0)}$ for $t \ge t_0$ . In particular, solutions that start at the same point are equal.

Solution

Plan. This is Gronwall’s inequality. Integrate the difference, bound with the Lipschitz condition, and solve the resulting inequality by multiplying by $e^{-Lt}$ .

Let $u(t) = \lVert x(t) - z(t) \rVert$ and $c = u(t_0)$ . By the integral equation and the norm-of-an-integral fact, $u(t) \le c + L\int_{t_0}^{t} u(s)\,ds$ . Let $w(t) = c + L\int_{t_0}^{t} u$ . Then $w' = Lu \le Lw$ , so $\left(e^{-L(t - t_0)}w\right)' = e^{-L(t - t_0)}(w' - Lw) \le 0$ . Hence $e^{-L(t - t_0)}w(t) \le w(t_0) = c$ , and $u \le w \le c\,e^{L(t - t_0)}$ . If $c = 0$ then $u = 0$ for $t \ge t_0$ . (For $t \le t_0$ apply the same argument with $t \mapsto -t$ .) $\blacksquare$

proveMATH 376 worksheet 18 and HW7 #2 style; new application

Show that for $\lvert \lambda \rvert < 1$ the equation $x = \lambda \cos x$ has exactly one real solution. Which theorem do you use, and what are $X$ , $T$ and $q$ ?

Solution

Plan. Banach on $X = \mathbb R$ with $T(x) = \lambda\cos x$ . $X = \mathbb R$ is complete (1.2). $T$ maps $\mathbb R$ into $\mathbb R$ . By the mean value theorem $\lvert T(x) - T(y) \rvert = \lvert \lambda \rvert\,\lvert \sin c \rvert\,\lvert x - y \rvert \le \lvert \lambda \rvert\,\lvert x - y \rvert$ , so $T$ is a contraction with $q = \lvert \lambda \rvert < 1$ . Banach gives exactly one fixed point, and the iterates $x_{n+1} = \lambda \cos x_n$ converge to it from any start. $\blacksquare$

stretch

Show that if $x$ and $z$ solve $x' = F(x)$ (an autonomous system with $F$ locally Lipschitz) and $x(s) = z(r)$ for some times $s, r$ , then $z(t) = x(t + s - r)$ for all $t$ . Conclude that an orbit of a planar autonomous system that touches itself at two different times is closed (periodic).

Solution

Plan. Show that a time shift of a solution is again a solution, then use uniqueness.

Let $\tilde z(t) = x(t + s - r)$ . Then $\tilde z'(t) = x'(t + s - r) = F(x(t + s - r)) = F(\tilde z(t))$ , because $F$ does not depend on $t$ , so $\tilde z$ is a solution. And $\tilde z(r) = x(s) = z(r)$ . So $\tilde z$ and $z$ solve the same IVP at time $r$ , and by the corollary of this chapter they are equal. If an orbit meets itself, $x(s) = x(r)$ with $s \ne r$ , then with $z = x$ we get $x(t) = x(t + s - r)$ for all $t$ : $x$ is periodic with period $\lvert s - r \rvert$ . $\blacksquare$ (This is why the closed loops in the phase portraits of 7.4 are exactly the periodic solutions, and why two orbits cannot cross.)

stretchApostol 6.1 style

Find all solutions of $y' = \sqrt{\lvert y \rvert}$ , $y(0) = 0$ , and say which hypothesis of the Picard–Lindelöf theorem fails.

Solution

For $y > 0$ : $\frac{y'}{\sqrt y} = 1$ gives $2\sqrt y = t - c$ , so $y = \frac{(t - c)^2}{4}$ for $t \ge c$ . For $y < 0$ the equation reads $y' = \sqrt{-y}$ , and the same computation gives $y = -\frac{(d - t)^2}{4}$ for $t \le d$ . Since $y' = \sqrt{\lvert y \rvert} \ge 0$ , every solution is non-decreasing, so a solution through $0$ can rest at $0$ for a while and then leave on either side. All the solutions are, for any $-\infty \le d \le 0 \le c \le \infty$ ,

y(t) = \begin{cases} -\frac{(d - t)^2}{4} & t \le d,\\ 0 & d \le t \le c,\\ \frac{(t - c)^2}{4} & t \ge c. \end{cases}

(At $t = c$ both one-sided derivatives are $0 = \sqrt 0$ , and the same at $t = d$ .) The right-hand side is continuous but not Lipschitz near $0$ : $\frac{\sqrt{\lvert y \rvert}}{\lvert y \rvert} = \lvert y \rvert^{-1/2}$ is unbounded. Existence holds (Peano), uniqueness fails.

Summary

An ordinary differential equation relates an unknown function of one variable to its derivatives. An initial value problem adds the value at one time. Every equation of order $n$ is a first-order system for $x = (y, y', \dots, y^{(n-1)})$ .
The IVP $y' = f(t, y)$ , $y(t_0) = y_0$ is the same as the integral equation $y(t) = y_0 + \int_{t_0}^{t} f(s, y(s))\,ds$ , by the fundamental theorem of calculus. A solution is a fixed point of the Picard operator $T$ .
$f$ satisfies a Lipschitz condition with constant $L$ if $\lVert f(t, x) - f(t, z) \rVert \le L\,\lVert x - z \rVert$ . A bounded derivative gives it; a corner is allowed; a vertical tangent ( $y^{2/3}$ at $0$ ) is not.
Picard–Lindelöf: continuous plus Lipschitz gives exactly one solution on $[t_0 - h, t_0 + h]$ with $h = \min(a, b/M, 1/(2L))$ . The proof: the space of curves inside the rectangle is complete, $T$ maps it into itself, and $T$ shrinks distances by $Lh \le \frac12$ . Picard iterates converge uniformly.
Two solutions that meet are equal, for as long as both exist. For linear systems the solution exists on the whole interval, because $T^m$ is a contraction for large $m$ .
Without Lipschitz, uniqueness fails ( $y' = 3y^{2/3}$ , infinitely many solutions). Without a global Lipschitz condition, a solution can blow up in finite time ( $y' = y^2$ gives $\frac{1}{1-t}$ ).
In practice: verify both the equation and the starting value, state which hypothesis you use, and remember the theorem is local. Euler’s method is Picard’s idea at finite step size, with error proportional to $h$ .

Watch and read

The same ideas explained by someone else. Use these when an explanation here does not land.

7.1 Existence and uniqueness for differential equations

The picture

The idea, in words

Definitions and theorems

Differential equations and starting values

Ordinary differential equation

Initial value problem

Higher order is first order in disguise

A higher-order equation is a first-order system

The integral equation

An IVP is an integral equation

The Lipschitz condition

Lipschitz condition

Derivative test for a Lipschitz condition

Picard iteration

Picard iteration

The Picard–Lindelöf theorem

Picard–Lindelöf theorem

The norm of an integral

Uniqueness on a long interval, and linear equations

Two solutions that meet are equal

Linear systems have solutions on the whole interval

When the hypotheses fail

Where it is used

Traps

Problems

Summary

Watch and read

Where this sits

Comes from

Leads to

Same idea elsewhere