Unit 7

Differential equations

Change described by equations, solved with linear algebra.

7 chapters written · 49 definitions · 68 theorems · builds on Unit 3, Unit 4, Unit 5

The story

Differential equations describe change. Existence and uniqueness (7.1) come from the fixed point theorem of Unit 4. Linear equations have solution sets that are vector spaces (7.2 and 7.3), so Unit 2 applies directly.

Systems $x' = Ax$ are solved by the matrix exponential, computed with eigenvalues (7.4). Power series solve the equations that have no closed form (7.5), and phase portraits show long-run behaviour without solving anything (7.6).

Chapters

The theorem chain

Every theorem in the unit, in the order it is proved. Each one is used by the ones after it. The label says whether it is on the 375/376 exams (T), an Apostol extra (A), or beyond both (B).

7.1 Existence and uniqueness for differential equations

Fact · T A higher-order equation is a first-order system Put $x = (y, y', \dots, y^{(n-1)})$ . Then $y^{(n)} = f(t, y, \dots, y^{(n-1)})$ is the same as $x' = F(t, x)$ for a vector function $F$ .
Theorem · T An IVP is an integral equation A continuous function $y$ solves $y' = f(t, y)$ , $y(t_0) = y_0$ on an interval exactly when $y(t) = y_0 + \int_{t_0}^{t} f(s, y(s))\,ds$ for every $t$ in it.
Theorem · B Derivative test for a Lipschitz condition If $f(t, x)$ is differentiable in the real variable $x$ on a region where $\lvert \partial f/\partial x \rvert \le L$ and the region contains the segment between any two positions at the same time, then $f$ satisfies a Lipschitz condition with constant $L$ .
Theorem · T Picard–Lindelöf theorem If $f$ is continuous and Lipschitz in $x$ on a rectangle around $(t_0, x_0)$ , the IVP has exactly one solution on a short interval around $t_0$ , and Picard iteration converges to it.
Fact · B The norm of an integral For a continuous curve $v : [a, b] \to \mathbb R^m$ , $\lVert \int_a^b v(s)\,ds \rVert \le \int_a^b \lVert v(s) \rVert\,ds$ .
Corollary · B Two solutions that meet are equal If $f$ is continuous and satisfies a Lipschitz condition on every closed rectangle inside its domain, then two solutions of the same IVP agree on every interval where both are defined.
Theorem · B Linear systems have solutions on the whole interval If $A(t)$ and $g(t)$ are continuous on an interval $J$ , then $x' = A(t)x + g(t)$ , $x(t_0) = x_0$ has exactly one solution, and it is defined on all of $J$ .

7.2 Linear differential equations

Fact · T A linear differential operator is a linear map $L(f + g) = Lf + Lg$ and $L(cf) = c\,Lf$ , so the solutions of $Ly = 0$ form the null space of a linear map.
Theorem · T Homogeneous and inhomogeneous solution sets The solutions of $Ly = 0$ form a vector space. If $Ly = R$ and $R$ is not the zero function, the solutions do NOT form a vector space: they are $y_p + \ker L$ , where $y_p$ is any one solution.
Theorem · T Existence and uniqueness for linear equations of order n For continuous coefficients and right-hand side on $J$ , any $t_0 \in J$ and any numbers $c_0, \dots, c_{n-1}$ , the equation $Ly = R$ has exactly one solution on all of $J$ with $y^{(k)}(t_0) = c_k$ for $k < n$ .
Theorem · T Dimension of the solution space of a linear ODE The solution space $\ker L$ of a homogeneous linear equation of order $n$ has dimension exactly $n$ .
Corollary · T General solution of a linear equation If $b_1, \dots, b_n$ is a basis of $\ker L$ and $y_p$ solves $Ly = R$ , then every solution is $y = y_p + c_1b_1 + \dots + c_nb_n$ .
Theorem · T Constant-coefficient operators multiply like polynomials If $A = P_A(D)$ and $B = P_B(D)$ then $AB = (P_AP_B)(D)$ . In particular $AB = BA$ .
Fact · T Derivative of a complex exponential For a complex number $r = \alpha + i\beta$ , $e^{rt} = e^{\alpha t}(\cos\beta t + i\sin\beta t)$ satisfies $D(e^{rt}) = re^{rt}$ .
Fact · T Shifting an exponential For a polynomial $p$ and a number $r$ : $(D - r)\big(p(t)e^{rt}\big) = p'(t)\,e^{rt}$ .
Theorem · T Basis for repeated roots The $m$ functions $e^{rt}, te^{rt}, \dots, t^{m-1}e^{rt}$ are a basis of the solutions of $(D - r)^my = 0$ .
Fact · T Kernel of a factor lies in the kernel of the product If $A = A_1A_2\cdots A_k$ are constant-coefficient operators, then $\ker A_j \subseteq \ker A$ for every $j$ .
Theorem · T Basis of solutions from the roots If $P_A(t) = \prod_{i=1}^{k}(t - r_i)^{m_i}$ with distinct roots $r_i$ , the $n = \sum m_i$ functions $t^je^{r_it}$ ( $i \le k$ , $0 \le j < m_i$ ) are a basis of the solutions of $Ay = 0$ .
Theorem · T Real solutions from complex roots If $A$ has real coefficients and $r = \alpha + i\beta$ ( $\beta \ne 0$ ) is a root of multiplicity $m$ , then so is $\bar r$ , and the $2m$ functions $t^je^{\alpha t}\cos\beta t$ and $t^je^{\alpha t}\sin\beta t$ ( $0 \le j < m$ ) replace the $2m$ complex solutions $t^je^{rt}$ and $t^je^{\bar rt}$ .

7.3 The Wronskian, variation of parameters and undetermined coefficients

Theorem · T A non-zero Wronskian proves independence If $W(f_1, \dots, f_n)(t_0) \ne 0$ at even one point $t_0$ of $J$ , then $f_1, \dots, f_n$ are linearly independent on $J$ .
Theorem · T Wronskian of solutions of a linear equation For $n$ solutions of $Ly = 0$ the following are equivalent: they are linearly independent; $W(t) \ne 0$ for every $t$ ; $W(t_0) \ne 0$ for some $t_0$ . So $W$ is either zero everywhere or non-zero everywhere.
Theorem · A Abel's formula If $b_1, \dots, b_n$ solve $y^{(n)} + a_{n-1}y^{(n-1)} + \dots = 0$ , their Wronskian satisfies $W' = -a_{n-1}W$ , so $W(t) = W(t_0)\exp\!\big(-\int_{t_0}^{t}a_{n-1}\big)$ .
Fact · T Wronskian of exponentials For distinct numbers $r_1, \dots, r_n$ , $W(e^{r_1t}, \dots, e^{r_nt}) = e^{(r_1 + \dots + r_n)t}\cdot V$ with $V = \det(r_i^{\,j})$ non-zero, so the exponentials are independent.
Theorem · T Wronskian test for power series If $f$ and $g$ are power series with radius of convergence $1$ , then $f, g$ are linearly independent on $(-1, 1)$ if and only if $W(f, g)$ is not identically zero.
Theorem · T Variation of parameters formula If $b_1, \dots, b_n$ is a basis of $\ker L$ with Wronskian matrix $\mathcal W$ , and $u' = \mathcal W^{-1}(0, \dots, 0, R)^{\mathsf T}$ , then $y_p = u_1b_1 + \dots + u_nb_n$ solves $Ly = R$ .
Theorem · A Shape of a particular solution If $R = q(t)e^{rt}$ with $q$ a polynomial of degree $d$ and $r$ is a root of $P_L$ of multiplicity $s$ ( $s = 0$ if $r$ is not a root), then $Ly = R$ has a particular solution $y_p = t^s\,h(t)\,e^{rt}$ with $h$ a polynomial of degree at most $d$ .

7.4 Systems of equations and the matrix exponential

Fact · T Eigenvectors give solutions If $Av = \lambda v$ with $v \ne O$ , then $x(t) = e^{\lambda t}v$ solves $x' = Ax$ .
Theorem · T The matrix norm is submultiplicative $\lVert AB\rVert \le \lVert A\rVert\,\lVert B\rVert$ , hence $\lVert A^k\rVert \le \lVert A\rVert^k$ .
Theorem · T Absolutely convergent matrix series converge If $\sum_k\lVert M_k\rVert$ converges as a series of numbers, then $\sum_kM_k$ converges in the space of matrices.
Theorem · T The matrix exponential always converges For every square matrix $M$ the series $\sum M^k/k!$ converges, and $\lVert e^M\rVert \le n - 1 + e^{\lVert M\rVert}$ .
Theorem · T The derivative of e^(tA) $\dfrac{d}{dt}e^{tA} = Ae^{tA} = e^{tA}A$ .
Theorem · T The law of exponents for one matrix $e^{0A} = I$ , $e^{(s+t)A} = e^{sA}e^{tA}$ , and $e^{tA}$ is invertible with inverse $e^{-tA}$ .
Theorem · T Solution of the linear system The IVP $x' = Ax$ , $x(a) = b$ has the unique solution $x(t) = e^{(t-a)A}b$ , defined for all real $t$ .
Theorem · T e^(A+B) equals e^A e^B when A and B commute If $AB = BA$ then $e^{A+B} = e^Ae^B$ . Without $AB = BA$ the equation can fail.
Corollary · T e^A is always invertible For every square matrix $A$ , $e^A$ is invertible and $(e^A)^{-1} = e^{-A}$ . Moreover $\det e^A = e^{\operatorname{tr}A}$ (a fact beyond the course).
Fact · T The exponential of a diagonal matrix $e^{t\operatorname{diag}(d_1, \dots, d_n)} = \operatorname{diag}(e^{d_1t}, \dots, e^{d_nt})$ .
Theorem · T Exponential of a diagonalizable matrix If $A = CDC^{-1}$ with $D$ diagonal, then $e^{tA} = Ce^{tD}C^{-1}$ .
Theorem · T Exponential of a rotation-scaling block For $A = \begin{bmatrix}a & -b\\b & a\end{bmatrix} = aI + bJ$ with $J = \begin{bmatrix}0 & -1\\1 & 0\end{bmatrix}$ : $e^{tA} = e^{at}\begin{bmatrix}\cos bt & -\sin bt\\\sin bt & \cos bt\end{bmatrix}$ .
Theorem · A Exponential of a 2 by 2 matrix For a $2 \times 2$ matrix with eigenvalues $\lambda, \mu$ : if $\lambda = \mu$ , $e^{tA} = e^{\lambda t}\big(I + t(A - \lambda I)\big)$ ; if $\lambda \ne \mu$ , $e^{tA} = e^{\lambda t}I + \frac{e^{\lambda t} - e^{\mu t}}{\lambda - \mu}(A - \lambda I)$ ; if $\lambda, \mu = a \pm ib$ , $e^{tA} = e^{at}\big(\cos bt\,I + \frac{\sin bt}{b}(A - aI)\big)$ .
Theorem · T Real and imaginary parts of a complex solution If $A$ is real and $Av = \lambda v$ with $\lambda = a + ib$ , $b \ne 0$ , and $v = u + iw$ with $u, w$ real vectors, then $x_1 = e^{at}(\cos bt\,u - \sin bt\,w)$ and $x_2 = e^{at}(\sin bt\,u + \cos bt\,w)$ are two independent real solutions of $x' = Ax$ .
Theorem · T Exponential of a scalar plus a nilpotent matrix If $A = \lambda I + N$ with $N^k = O$ , then $e^{tA} = e^{\lambda t}\Big(I + tN + \frac{t^2N^2}{2!} + \dots + \frac{t^{k-1}N^{k-1}}{(k-1)!}\Big)$ .
Theorem · A Putzer's method With eigenvalues $\lambda_1, \dots, \lambda_n$ listed with multiplicity and $P_0 = I$ , $P_k = (A - \lambda_1I)\cdots(A - \lambda_kI)$ , one has $e^{tA} = \sum_{k=0}^{n-1}r_{k+1}(t)P_k$ , where $r_1' = \lambda_1r_1$ , $r_1(0) = 1$ and $r_{k+1}' = \lambda_{k+1}r_{k+1} + r_k$ , $r_{k+1}(0) = 0$ .
Theorem · T Variation of constants for systems The IVP $x' = Ax + g(t)$ , $x(a) = b$ has the unique solution $x(t) = e^{(t-a)A}b + \int_a^te^{(t-s)A}g(s)\,ds$ .
Theorem · T The Neumann series If $\lVert A\rVert < 1$ for the matrix norm of this chapter, then $I - A$ is invertible and $(I - A)^{-1} = \sum_{k\ge0}A^k$ .
Theorem · B Classification of planar linear systems

7.5 Power series solutions, Legendre and Bessel

Theorem · T Matching coefficients If a power series $\sum c_n(x - x_0)^n$ equals $0$ at every point of an interval around $x_0$ , then every coefficient $c_n$ is $0$ .
Theorem · A Series solutions at an ordinary point If $p$ and $q$ are analytic on $(x_0 - r, x_0 + r)$ , then $y'' + py' + qy = 0$ has two independent solutions that are power series in $x - x_0$ converging on the whole interval. The coefficients come from a recurrence, with $a_0$ and $a_1$ free.
Theorem · A Recurrence for Legendre's equation Series solutions $y = \sum a_nx^n$ of Legendre's equation satisfy $a_{n+2} = \dfrac{(n-\nu)(n+\nu+1)}{(n+1)(n+2)}\,a_n$ . They converge for $\lvert x \rvert < 1$ , and they stop (are polynomials) when $\nu$ is an integer of the right parity.
Theorem · A Legendre polynomials solve Legendre's equation $P_n$ has degree $n$ and satisfies $(1 - x^2)P_n'' - 2xP_n' + n(n+1)P_n = 0$ and $P_n(1) = 1$ .
Theorem · A Orthogonality of the Legendre polynomials $\int_{-1}^{1}P_m(x)P_n(x)\,dx = 0$ for $m \ne n$ , and $\int_{-1}^{1}P_n(x)^2\,dx = \dfrac{2}{2n+1}$ .
Corollary · A Expansion in Legendre polynomials $P_n$ is orthogonal to every polynomial of degree less than $n$ . A polynomial $f$ of degree at most $N$ equals $\sum_{k=0}^{N}c_kP_k$ with $c_k = \frac{2k+1}{2}\int_{-1}^{1}fP_k\,dx$ .
Theorem · A The Frobenius recurrence The Frobenius series $x^s\sum a_nx^n$ with $a_0 = 1$ solves $x^2y'' + xPy' + Qy = 0$ exactly when $F(s) = 0$ and, for every $n \ge 1$ , $F(s+n)\,a_n = -\sum_{k=0}^{n-1}\big[(s+k)p_{n-k} + q_{n-k}\big]a_k$ .
Theorem · A The Frobenius theorem At a regular singular point with indicial roots $\alpha_1 \ge \alpha_2$ , the larger root always gives a Frobenius series solution $u_1$ . If $\alpha_1 - \alpha_2$ is not a whole number, the smaller root gives a second one. If the roots are equal, or differ by a whole number, a second solution contains $u_1\ln x$ .
Theorem · A J0 solves Bessel's equation of order zero The series $J_0$ converges for every real $x$ and solves $x^2y'' + xy' + x^2y = 0$ . The indicial equation is $s^2 = 0$ (a double root), so a second solution contains $\ln x$ .

7.6 Phase portraits, stability and linearization

Fact · B Orbits do not cross Two orbits of an autonomous system either coincide or never meet. A solution that comes back to its starting point is periodic, and its orbit is a closed curve.
Theorem · B Classification of the origin by trace and determinant
Theorem · B Stability of a linear system For $x' = Ax$ with $A$ real $n \times n$ : the origin is asymptotically stable if every eigenvalue has negative real part (and then $\lVert e^{tA}\rVert \le Ce^{-\mu t}$ ); unstable if some eigenvalue has positive real part. If all real parts are $\le 0$ it is stable exactly when every eigenvalue with real part $0$ has only $1 \times 1$ Jordan blocks.
Theorem · B Hartman–Grobman theorem Near a hyperbolic equilibrium, a nonlinear system has the same phase portrait as its linearization, up to a continuous change of coordinates that preserves the direction of time.
Theorem · B Linearization and stability If every eigenvalue of $J = DF(x^*)$ has negative real part, then $x^*$ is asymptotically stable for the nonlinear system. If some eigenvalue has positive real part, it is unstable.
Theorem · B The Lotka–Volterra conservation law For $x' = x(a - by)$ , $y' = y(dx - c)$ with positive constants, $H = dx - c\ln x + by - a\ln y$ is constant along solutions in the open first quadrant. Every orbit there is a closed curve around the equilibrium $(c/d,\ a/b)$ , and the average of $x$ and of $y$ over a period equal its equilibrium value.
Theorem · B Lyapunov's direct method If there is a Lyapunov function near $x^*$ , then $x^*$ is stable. If there is a strict one, $x^*$ is asymptotically stable.

7.7 Applications to springs, circuits, epidemics and qubits

Theorem · B Normal-mode decomposition If $M$ is symmetric positive definite with orthonormal eigenvectors $v_1, \dots, v_n$ and eigenvalues $\omega_j^2$ , then every solution of $x'' = -Mx$ is $x(t) = \sum_j\big(a_j\cos\omega_jt + b_j\sin\omega_jt\big)v_j$ with $a_j = v_j\cdot x(0)$ and $b_j = v_j\cdot x'(0)/\omega_j$ .
Fact · B A passive circuit loses energy With no source, the stored energy $W = \frac12LI^2 + \frac{q^2}{2C}$ satisfies $\dot W = -RI^2 \le 0$ . With a source, $\dot W = IE - RI^2$ .
Theorem · T Steady-state response of the driven RLC circuit For $E(t) = E_0\cos\omega t$ , every solution is a decaying transient plus the steady state $I_{ss}(t) = \dfrac{E_0}{\lvert Z\rvert}\cos(\omega t - \varphi)$ , where $Z = R + i\left(\omega L - \dfrac{1}{\omega C}\right)$ is the impedance.
Theorem · B The epidemic threshold
Theorem · B Peak and final size of an epidemic Along every solution of the SIR system, $i + s - \dfrac1{R_0}\ln s$ is constant. Hence the infection peaks when $s = 1/R_0$ at $i_{\max} = i_0 + s_0 - \dfrac{1 + \ln(R_0s_0)}{R_0}$ , and the epidemic ends with $i \to 0$ and $s \to s_\infty$ where $\ln\dfrac{s_0}{s_\infty} = R_0(s_0 + i_0 - s_\infty)$ .
Theorem · B Unitary evolution The solution of $i\hbar\psi' = H\psi$ is $\psi(t) = U(t)\psi(0)$ with $U(t) = e^{-iHt/\hbar}$ . For Hermitian $H$ the matrix $U(t)$ is unitary, so $\lVert\psi(t)\rVert = \lVert\psi(0)\rVert$ for all $t$ : total probability stays $1$ .
Theorem · B The Rabi formula For $H = \frac\hbar2(\delta\sigma_z + \Omega\sigma_x)$ and $\Omega_R = \sqrt{\delta^2 + \Omega^2}$ : $e^{-iHt/\hbar} = \cos\frac{\Omega_Rt}{2}\,I - i\sin\frac{\Omega_Rt}{2}\cdot\frac{\delta\sigma_z + \Omega\sigma_x}{\Omega_R}$ . Starting in level 1, the probability of level 2 is $\dfrac{\Omega^2}{\Omega_R^2}\sin^2\dfrac{\Omega_Rt}{2}$ .

Terms this unit defines

Unit review

A mixed set of 12 quick checks drawn at random from the 53 in this unit's chapters. Each one says which chapter it comes from, so a miss tells you what to reread. Press "New set" for another mix.

Where it leads

Units that build on this one: Unit 15, Optimisation, dynamics and computation.