7.4 Systems of equations and the matrix exponential

The system x' = Ax is solved by x(t) = e^(tA) x(0), where the matrix exponential is the exponential series with a matrix in place of the number. It always converges, and it is computed by diagonalizing, by splitting A into a scalar part and a nilpotent part (Jordan form), or by Putzer's method. The eigenvalues also decide the shape of the solution curves.

In 7.2 one equation $y' = ry$ was solved by $y = y(0)e^{rt}$ . A system of such equations, with a matrix in place of $r$ , deserves the same answer: $x(t) = e^{tA}x(0)$ . This chapter makes sense of $e^{tA}$ (an infinite series of matrices), proves it solves the system, and teaches the ways to compute it. Along the way the eigenvalues of 2.8 and the Jordan blocks of 3.6 return, now with a job: they tell us how the solution moves.

Before you start

You need eigenvectors and diagonalization, the Jordan form and the Cayley–Hamilton theorem, norms and absolute convergence, and the existence theorem of 7.1. Three quick checks:

Quick checkIf

A = CDC^{-1}

with

D

diagonal, what is

A^3

Quick checkFor which real numbers

x

does the series

\sum_{k=0}^{\infty}\frac{x^k}{k!}

converge?

Quick checkThe equation

y'' + 4y = 0

is written as

x' = Ax

with

x = (y, y')

. Which is

A

The picture

For a $2 \times 2$ matrix $A$ , a solution $x(t)$ of $x' = Ax$ is a point moving in the plane, and its path is a curve. The velocity of the point at the position $x$ is $Ax$ . The picture draws many such curves at once. The four sliders are the entries $a, b, c, d$ of $A = \begin{bmatrix}a & b\\c & d\end{bmatrix}$ . The arrows show the direction of time. The dashed purple lines are the eigendirections (when they are real): the lines through the origin spanned by real eigenvectors of $A$ . Drag the orange dot to start a new curve.

Four words, used loosely for now (chapter 7.6 will define them carefully). The curve traced by one solution is its orbit, and the plane with many orbits drawn in is the phase portrait. An equilibrium is a point where the velocity is zero, so a solution that starts there never moves; for $x' = Ax$ the origin is always one, because $A\,O = O$ . In this chapter an equilibrium is called stable when every solution tends to it as $t \to \infty$ , and unstable when solutions move away from it. (The careful definition of stable is weaker, and under it a centre also counts as stable. That is the point of 7.6.)

Go through the presets. Saddle: one eigenvalue positive and one negative; curves come in along one eigendirection and leave along the other. Stable node and unstable node: both eigenvalues of the same sign, so everything runs into or out of the origin. Spirals: complex eigenvalues $\alpha \pm i\beta$ turn the curves around the origin; they shrink if $\alpha < 0$ and grow if $\alpha > 0$ . Centre: pure imaginary eigenvalues give closed loops. One eigenvector: a repeated eigenvalue with only one eigendirection bends the curves round it. The last preset is the matrix of the 376 final’s Problem 5(b): trace $4$ , determinant $7$ , an unstable spiral. The theorems below explain every picture.

Where this came from· Laguerre's series and Putzer's shortcut

The power series $I + A + A^2/2! + \cdots$ for a matrix was introduced by Edmond Laguerre in 1867, as Nicholas Higham’s survey of the matrix exponential records. The point is the pattern: $y' = ay$ has the solution $e^{at}y(0)$ , and $x' = Ax$ has $e^{tA}x(0)$ . Textbooks computed $e^{tA}$ by putting $A$ into Jordan form, which is delicate: a tiny change of $A$ can change the Jordan form completely (chapter 3.6). In 1966 E. J. Putzer published “Avoiding the Jordan canonical form in the discussion of linear systems with constant coefficients” in the American Mathematical Monthly, volume 73, pages 2 to 7. His method writes $e^{tA}$ as a combination of the matrices $I$ , $A - \lambda_1I$ , $(A - \lambda_1I)(A - \lambda_2I)$ , … using only the eigenvalues; Apostol presents it in section 7.13, and we prove it below.

The idea, in words

Why· Three ideas: the series, the flow, the eigen-directions

The series. The number $e^{at}$ is $\sum (at)^k/k!$ , the solution of $x' = ax$ with $x(0) = 1$ . Replace $a$ by a matrix $A$ and $1$ by the identity matrix. Matrices can be multiplied and added, so $\sum t^kA^k/k!$ makes sense term by term; the only question is whether it converges, and it does, for every matrix and every $t$ , because $k!$ grows faster than any power of $\lVert A\rVert$ . The flow. $x(t) = e^{tA}x(0)$ says: to find where the point is at time $t$ , apply the matrix $e^{tA}$ to where it started. The matrices $e^{tA}$ form a one-parameter family with $e^{(s+t)A} = e^{sA}e^{tA}$ : flowing for $s$ and then for $t$ is flowing for $s + t$ . The eigen-directions. If $Av = \lambda v$ , then $e^{tA}v = e^{\lambda t}v$ (every power $A^k$ multiplies $v$ by $\lambda^k$ ). So along an eigenvector the point moves by the scalar rule $e^{\lambda t}$ . When there is a basis of eigenvectors, this solves everything; when there is not, a nilpotent part adds powers of $t$ (the shear $e^t\begin{bmatrix}1 & t\\0 & 1\end{bmatrix}$ of the repeated-root case in 7.2).

Definitions and theorems

Systems of differential equations

Definition T

System of differential equations

A system of differential equations (first order) is $x'(t) = F(t, x(t))$ with $x(t) \in \mathbb R^n$ (or $\mathbb C^n$ ); this is the form into which every higher-order equation is turned (7.1). The system is linear if $F(t, x) = A(t)x + g(t)$ with a matrix function $A$ and a vector function $g$ ; it is homogeneous if $g = 0$ . In this chapter $A$ is a constant matrix. Technau writes the general linear system as $y' = q + Py$ and the constant-coefficient homogeneous case as $y' = Ay$ .

In one lineSeveral unknown functions $x_1(t), \dots, x_n(t)$ whose derivatives are given in terms of $t$ and the unknowns; in vector form $x' = F(t, x)$ . It is linear if $x' = A(t)x + g(t)$ and has constant coefficients if $A$ does not depend on $t$ .

Why it existsMany quantities interact (two populations, two currents, position and velocity). A system tracks them together as one point moving in $\mathbb R^n$ .

The existence and uniqueness theorem of 7.1 for linear systems gives immediately: for every starting vector $b$ and starting time $a$ , the IVP $x' = Ax$ , $x(a) = b$ has exactly one solution, and it is defined for all $t \in \mathbb R$ . The task is to write it down.

Fact T MATH 376 worksheet 25 #6; Apostol 7.10

Eigenvectors give solutions

If $Av = \lambda v$ , then $x(t) = e^{\lambda t}v$ satisfies $x' = Ax$ (for complex $\lambda$ and $v$ too). Hence if $A$ has a basis of eigenvectors $v_1, \dots, v_n$ with eigenvalues $\lambda_1, \dots, \lambda_n$ , every solution is $x(t) = c_1e^{\lambda_1t}v_1 + \dots + c_ne^{\lambda_nt}v_n$ , where $c = (c_1, \dots, c_n)$ are the coordinates of $x(0)$ in that basis.

In one lineIf $Av = \lambda v$ with $v \ne O$ , then $x(t) = e^{\lambda t}v$ solves $x' = Ax$ .

Why it is trueDifferentiating $e^{\lambda t}v$ brings down $\lambda$ , and $\lambda v = Av$ .

ProofApostol 7.10

The idea: Differentiate; then write the starting vector in the eigenbasis and use linearity and uniqueness.

x' = \lambda e^{\lambda t}v

\frac{d}{dt}e^{\lambda t} = \lambda e^{\lambda t}

and

v

is constant.

Ax = e^{\lambda t}Av = \lambda e^{\lambda t}v = x'

Av = \lambda v

, and

A

commutes with the scalar

e^{\lambda t}

\sum c_je^{\lambda_jt}v_j

is a solution with the right starting vector.

The equation is linear, so sums and multiples of solutions are solutions. At

t = 0

the combination equals

\sum c_jv_j = x(0)

So it is the solution.

Uniqueness (7.1): a linear IVP has exactly one solution.

\blacksquare

Worked example MATH 376 worksheet 28 #4

Turn y'' + 4y = 0 into a system and compute e^(tA) from the series.

Solution.

Write the system. With $x = (y, y')$ : $x' = Ax$ and $A = \begin{bmatrix}0 & 1\\-4 & 0\end{bmatrix}$ . The companion matrix of 7.2.
Compute the powers. $A^2 = \begin{bmatrix}-4 & 0\\0 & -4\end{bmatrix} = -4I$ . So $A^{2k} = (-4)^kI$ and $A^{2k+1} = (-4)^kA$ . Multiply $A$ by itself once; the pattern then repeats.
Split the series into even and odd powers. $e^{tA} = \sum_k\frac{(-4)^kt^{2k}}{(2k)!}I + \sum_k\frac{(-4)^kt^{2k+1}}{(2k+1)!}A$ . Even $k$ in $\sum t^kA^k/k!$ give multiples of $I$ ; odd $k$ give multiples of $A$ .
Recognise the sums. $(-4)^kt^{2k} = (-1)^k(2t)^{2k}$ , so the first sum is $\cos2t$ and the second is $\frac12\sin2t$ . The series for cosine and sine (1.6).
Assemble. $e^{tA} = \cos2t\,I + \frac12\sin2t\,A = \begin{bmatrix}\cos2t & \frac12\sin2t\\-2\sin2t & \cos2t\end{bmatrix}$ . The first row applied to $(y(0), y'(0))$ gives $y = y(0)\cos2t + \frac{y'(0)}{2}\sin2t$ , the answer from 7.2.

Answer. $e^{tA} = \begin{bmatrix}\cos2t & \frac12\sin2t\\-2\sin2t & \cos2t\end{bmatrix}$ .

Norms and convergence of matrix series

To say that a series of matrices converges we need a notion of distance for matrices. Technau’s course uses the simplest one.

Definition T

Matrix norm

For an $n \times n$ matrix $A = (a_{ij})$ define the matrix norm $\lVert A\rVert = \sum_{i,j=1}^{n}\lvert a_{ij}\rvert$ (the $1$ -norm of 4.1 on $\mathbb R^{n^2}$ , called $\lVert A\rVert_1$ in Technau’s notes). It satisfies the three norm axioms because it is a norm on $\mathbb R^{n^2}$ . Any other norm with $\lVert AB\rVert \le \lVert A\rVert\lVert B\rVert$ , such as the operator norm of 3.5, would do equally well. Distance and convergence of matrices are those of this norm; since all norms on $\mathbb R^{n^2}$ are equivalent, the choice does not change which sequences converge.

In one lineFor this chapter, $\lVert A \rVert = \sum_{i,j}\lvert a_{ij}\rvert$ : the sum of the absolute values of all entries. It is a norm, and it satisfies $\lVert AB\rVert \le \lVert A\rVert\,\lVert B\rVert$ .

Why it existsIt measures the size of a matrix by one number, makes the set of $n \times n$ matrices a complete normed space, and has the extra property $\lVert AB \rVert \le \lVert A\rVert\lVert B\rVert$ that powers need.

Theorem T MATH 376 homework 10 #2; Technau, April 15 (8.3)

The matrix norm is submultiplicative

For $n \times n$ matrices $A, B$ : $\lVert AB\rVert \le \lVert A\rVert\,\lVert B\rVert$ . Consequently $\lVert A^k\rVert \le \lVert A\rVert^k$ for all $k \ge 1$ .

In one line $\lVert AB\rVert \le \lVert A\rVert\,\lVert B\rVert$ , hence $\lVert A^k\rVert \le \lVert A\rVert^k$ .

Why it is trueEach entry of $AB$ is a sum of products; bounding by absolute values and then adding extra non-negative terms gives the product of the two norms.

ProofTechnau, April 15

The idea: Triangle inequality inside each entry of the product, then enlarge the sum by allowing independent indices.

\lVert AB\rVert = \sum_{i,j}\Big\lvert\sum_ka_{ik}b_{kj}\Big\rvert \le \sum_{i,j,k}\lvert a_{ik}\rvert\,\lvert b_{kj}\rvert

(AB)_{ij} = \sum_ka_{ik}b_{kj}

, and the absolute value of a sum is at most the sum of the absolute values.

\le \sum_{i,k}\sum_{l,j}\lvert a_{ik}\rvert\,\lvert b_{lj}\rvert

Adding the extra non-negative terms

\lvert a_{ik}\rvert\lvert b_{lj}\rvert

with

l \ne k

only makes the sum larger.

= \Big(\sum_{i,k}\lvert a_{ik}\rvert\Big)\Big(\sum_{l,j}\lvert b_{lj}\rvert\Big) = \lVert A\rVert\,\lVert B\rVert

The double sum factors into two independent double sums.

\lVert A^k\rVert \le \lVert A\rVert^k

Induction on

k

\lVert A^{k+1}\rVert = \lVert A\cdot A^k\rVert \le \lVert A\rVert\,\lVert A^k\rVert \le \lVert A\rVert^{k+1}

\blacksquare

Worked example MATH 376 worksheet 24 #1

Compute the matrix norm of A = [[1, −2], [3, 0]] and B = [[1, 1, 0], [−1, 2, 4]].

Solution.

Add the absolute values of all entries of $A$ . $1 + 2 + 3 + 0 = 6$ . The definition: every entry counts, no maximum and no row sums.
Do the same for $B$ . $1 + 1 + 0 + 1 + 2 + 4 = 9$ . The definition works for any shape of matrix; submultiplicativity is for square ones.

Answer. $\lVert A\rVert = 6$ and $\lVert B\rVert = 9$ . (This is not the operator norm and not the largest column sum.)

Your answerWhat is

\lVert A\rVert = \sum_{i,j}\lvert a_{ij}\rvert

for

A = \begin{bmatrix}1 & -2\\3 & 0\end{bmatrix}

Theorem T MATH 376 worksheet 24 #2

Absolutely convergent matrix series converge

Let $M_0, M_1, \dots$ be $n \times n$ matrices with $\sum_k\lVert M_k\rVert < \infty$ . Then the partial sums $S_N = \sum_{k \le N}M_k$ converge to a matrix $S$ , and $\lVert S\rVert \le \sum_k\lVert M_k\rVert$ .

In one lineIf $\sum_k\lVert M_k\rVert$ converges as a series of numbers, then $\sum_kM_k$ converges in the space of matrices.

Why it is trueThe partial sums of the matrix series are within the tail of the number series of each other, and the space of matrices is complete.

ProofMATH 376 worksheet 24 #2

The idea: Show the partial sums are Cauchy using the triangle inequality, then apply completeness of R^(n^2).

For

m > n

\lVert S_m - S_n\rVert = \big\lVert\sum_{k=n+1}^{m}M_k\big\rVert \le \sum_{k=n+1}^{m}\lVert M_k\rVert

Triangle inequality for the norm, applied repeatedly.

So given

\varepsilon > 0

there is

N

with

\lVert S_m - S_n\rVert < \varepsilon

for

m > n > N

(S_N)

is Cauchy.

The number series

\sum\lVert M_k\rVert

converges, so its partial sums are Cauchy (1.2): the tails

\sum_{k>n}\lVert M_k\rVert

tend to

0

S_N \to S

for some matrix

S

\mathbb R^{n^2}

with this norm is complete (4.2), so every Cauchy sequence converges.

Passing to the limit,

\lVert S\rVert \le \sum_k\lVert M_k\rVert

\lVert S_N\rVert \le \sum_{k\le N}\lVert M_k\rVert

and norms are continuous.

\blacksquare

The matrix exponential

Definition T

Matrix exponential

For an $n \times n$ matrix $M$ (real or complex) the matrix exponential is $e^M = \sum_{k=0}^{\infty}\frac{1}{k!}M^k$ , where $M^0 = I$ . For a real number $t$ and a matrix $A$ , $e^{tA} = \sum_k\frac{t^kA^k}{k!}$ .

In one lineFor a square matrix $M$ , $e^M = \sum_{k=0}^{\infty}\frac{M^k}{k!} = I + M + \frac{M^2}{2!} + \cdots$ .

Why it existsIt is the exponential function applied to a matrix, defined by the same series, and it is the solution operator of the system $x' = Ax$ .

Theorem T Technau, April 15 and 17 (Lemma 8.3); MATH 376 worksheet 24 #3; Apostol 7.5

The matrix exponential always converges

For every $n \times n$ matrix $M$ , the series defining $e^M$ converges, and $\lVert e^M\rVert \le n - 1 + e^{\lVert M\rVert}$ .

In one lineFor every square matrix $M$ the series $\sum M^k/k!$ converges, and $\lVert e^M\rVert \le n - 1 + e^{\lVert M\rVert}$ .

Why it is trueCompare with the number series for $e^{\lVert M \rVert}$ , using $\lVert M^k\rVert \le \lVert M\rVert^k$ for $k \ge 1$ . The first term is the identity, whose norm is $n$ , not $1$ .

ProofTechnau, April 17; the constant corrects the worksheet

The idea: Bound the norm of each term with k at least 1 by the term of the number series for e to the power of the norm, treat the first term separately, then apply absolute convergence.

For

k \ge 1

\big\lVert\frac{M^k}{k!}\big\rVert \le \frac{\lVert M\rVert^k}{k!}

Submultiplicativity:

\lVert M^k\rVert \le \lVert M\rVert^k

for

k \ge 1

With

r = \lVert M\rVert

\sum_{k \ge 0}\big\lVert\frac{M^k}{k!}\big\rVert \le n + \sum_{k\ge1}\frac{\lVert M\rVert^k}{k!} = n - 1 + e^{\lVert M\rVert} < \infty

The number series

\sum\frac{r^k}{k!}

converges for every real

r

(ratio test, 1.6), and its sum is

e^r

. The

k = 0

term of the matrix series is

I

, with

\lVert I\rVert = n

(the sum of the absolute values of the

n

ones).

\sum M^k/k!

converges and

\lVert e^M\rVert \le n - 1 + e^{\lVert M\rVert}

\blacksquare

The previous theorem on absolutely convergent matrix series, with

M_k = \frac{M^k}{k!}

.The worksheet states the bound as e to the norm. That is false for n at least 2: for M = 0 the left side is the norm of I, which is n, and the right side is 1.

Watch out· e to the power of a matrix is not the matrix of exponentials of entries

$e^{tA}$ is NOT the matrix $\big[e^{ta_{ij}}\big]$ . For $A = \begin{bmatrix}0 & -1\\1 & 0\end{bmatrix}$ the entrywise exponentials would have entries $e^0 = 1$ and $e^{\pm t}$ , while the true $e^{tA}$ is the rotation matrix with entries $\cos t$ and $\pm\sin t$ . The only case where the two agree is a diagonal matrix, and then only on the diagonal.

Theorem T Technau, April 17 (Proposition 8.4); MATH 376 worksheet 24 #4; Apostol 7.6

The derivative of e^(tA)

For every square matrix $A$ and all real $t$ : $\frac{d}{dt}e^{tA} = Ae^{tA} = e^{tA}A$ , where the derivative of a matrix is taken entry by entry.

In one line $\dfrac{d}{dt}e^{tA} = Ae^{tA} = e^{tA}A$ .

Why it is trueEach entry of $e^{tA}$ is a power series in $t$ with infinite radius of convergence, so it can be differentiated term by term, and the shifted series is $A$ times the original.

ProofTechnau, April 17

The idea: The (i,j) entry is a power series in t whose coefficients are bounded by those of an exponential series. Differentiate it term by term (1.6), shift the index, and recognise A times the series.

\big(e^{tA}\big)_{ij} = \sum_{k\ge0}\alpha_kt^k

The

(i,j)

entry of

t^kA^k/k!

\alpha_kt^k

with

\alpha_k = (A^k)_{ij}/k!

The power series has radius of convergence

\infty

A single entry is at most the norm:

\lvert\alpha_k\rvert \le \frac{\lVert A^k\rVert}{k!} \le \frac{\lVert A\rVert^k}{k!}

. The comparison test then gives absolute convergence for every

t

\frac{d}{dt}\sum_k\alpha_kt^k = \sum_{k\ge1}k\alpha_kt^{k-1} = \sum_{k\ge1}\frac{(A^k)_{ij}}{(k-1)!}t^{k-1}

Power series may be differentiated term by term inside the radius of convergence (1.6 and differentiating a uniform limit).

= \Big(A\sum_{m\ge0}\frac{t^mA^m}{m!}\Big)_{ij} = \big(Ae^{tA}\big)_{ij}

Put

m = k - 1

, and use

A^{m+1} = A\cdot A^m

, so that

(A^{m+1})_{ij} = (A\cdot A^m)_{ij}

\frac{d}{dt}e^{tA} = Ae^{tA} = e^{tA}A

Also

A^{m+1} = A^m\cdot A

, so the same computation gives

e^{tA}A

\blacksquare

Proof workshopRebuild the proof that the derivative of e^(tA) is A e^(tA)Technau, April 17

Tap the lines in the order a proof would write them. Each line can only use what is already above it.

Theorem T MATH 376 worksheet 24 #5; Apostol 7.8

The law of exponents for one matrix

For every square $A$ and all real $s, t$ : (a) $e^{0A} = I$ ; (b) $e^{(s+t)A} = e^{sA}e^{tA}$ ; (c) $e^{tA}e^{-tA} = I$ , so $(e^{tA})^{-1} = e^{-tA}$ .

In one line $e^{0A} = I$ , $e^{(s+t)A} = e^{sA}e^{tA}$ , and $e^{tA}$ is invertible with inverse $e^{-tA}$ .

Why it is trueBoth sides of the law solve the same matrix differential equation with the same starting matrix, and the solution is unique.

ProofMATH 376 worksheet 24 #5

The idea: (a) is the series at t = 0. For (b), fix s: the two sides are functions of t satisfying X' = AX with X(0) = e^{sA}, so they are equal by uniqueness (7.1). (c) is (b) with s = -t.

(a)

e^{0A} = I

Every term with

k \ge 1

has a factor

0^k = 0

; the

k = 0

term is

I

(b) Let

F(t) = e^{(t+s)A}

. Then

F' = AF

and

F(0) = e^{sA}

The derivative theorem and the chain rule:

\frac{d}{dt}e^{(t+s)A} = Ae^{(t+s)A}

Let

G(t) = e^{tA}e^{sA}

. Then

G' = AG

and

G(0) = e^{sA}

The product rule and the derivative theorem:

G' = Ae^{tA}e^{sA} = AG

, and

G(0) = Ie^{sA}

F = G

for all

t

Each column of

F

and of

G

solves the vector IVP

x' = Ax

with the same starting vector. Linear systems have exactly one solution for each starting vector (7.1).

(c)

e^{tA}e^{-tA} = e^{0A} = I

\blacksquare

Put

s = -t

in (b) and use (a).

Theorem T Technau, April 17 (Theorem 8.6); Apostol 7.9

Solution of the linear system

Let $A$ be an $n \times n$ constant matrix, $b \in \mathbb R^n$ and $a \in \mathbb R$ . Then $x(t) = e^{(t-a)A}b$ is the unique solution of $x' = Ax$ , $x(a) = b$ on $\mathbb R$ .

In one lineThe IVP $x' = Ax$ , $x(a) = b$ has the unique solution $x(t) = e^{(t-a)A}b$ , defined for all real $t$ .

Why it is trueThe derivative theorem shows it is a solution, and uniqueness comes from the existence theorem of 7.1.

ProofTechnau, April 17

The idea: Differentiate with the chain rule and evaluate at a. Uniqueness is the linear existence theorem of 7.1.

x'(t) = Ae^{(t-a)A}b = Ax(t)

Chain rule and the derivative theorem:

\frac{d}{dt}e^{(t-a)A} = Ae^{(t-a)A}

x(a) = e^{0A}b = b

e^{0A} = I

So it is the only solution.

A linear system with continuous coefficients has exactly one solution for each starting vector (7.1).

\blacksquare

Theorem T MATH 376 homework 10 #4; Apostol 7.8

e^(A+B) equals e^A e^B when A and B commute

(a) If $AB = BA$ , then $e^{t(A+B)} = e^{tA}e^{tB}$ for all $t$ , in particular $e^{A+B} = e^Ae^B$ . (b) For $A = \begin{bmatrix}1 & 0\\0 & 0\end{bmatrix}$ and $B = \begin{bmatrix}0 & 1\\0 & 0\end{bmatrix}$ one has $e^Ae^B = \begin{bmatrix}e & e\\0 & 1\end{bmatrix}$ but $e^{A+B} = \begin{bmatrix}e & e - 1\\0 & 1\end{bmatrix}$ .

In one lineIf $AB = BA$ then $e^{A+B} = e^Ae^B$ . Without $AB = BA$ the equation can fail.

Why it is trueBoth sides, as functions of $t$ in $e^{tA}e^{tB}$ and $e^{t(A+B)}$ , solve the same differential equation with the same starting matrix.

ProofMATH 376 homework 10 #4

The idea: (a) Show e^(tA) commutes with B, differentiate the product e^(tA) e^(tB), and use uniqueness. (b) Compute both sides.

Be^{tA} = e^{tA}B

AB = BA

gives

BA^k = A^kB

for every

k

by induction. Multiplication by

B

is continuous, so it passes through the convergent sum.

Let

F(t) = e^{tA}e^{tB}

. Then

F'(t) = Ae^{tA}e^{tB} + e^{tA}Be^{tB}

Product rule and the derivative theorem:

F' = Ae^{tA}e^{tB} + e^{tA}Be^{tB}

= AF + Be^{tA}e^{tB} = (A + B)F

Move

B

to the left across

e^{tA}

by the first step.

F(0) = I

, so

F(t) = e^{t(A+B)}

e^{t(A+B)}

solves the same equation

X' = (A + B)X

with

X(0) = I

. Uniqueness (7.1) applied to each column.

Now the example.

(b)

A^k = A

for

k \ge 1

gives

e^A = I + (e - 1)A = \operatorname{diag}(e, 1)

B^2 = 0

gives

e^B = I + B

. The product is

\operatorname{diag}(e, 1)\begin{bmatrix}1 & 1\\0 & 1\end{bmatrix} = \begin{bmatrix}e & e\\0 & 1\end{bmatrix}

The two products differ in the top-right entry:

e

against

e - 1

A + B = \begin{bmatrix}1 & 1\\0 & 0\end{bmatrix}

satisfies

(A + B)^2 = A + B

, so

(A+B)^k = A + B

for

k \ge 1

and

e^{A+B} = I + (e - 1)(A + B) = \begin{bmatrix}e & e - 1\\0 & 1\end{bmatrix}

. And

AB = B \ne 0 = BA

\blacksquare

Proof workshopRebuild: e^(A+B) = e^A e^B when AB = BAMATH 376 homework 10 #4

Each line of the proof is shown in order. Choose the reason that justifies it, then press Check.

Since $AB = BA$ , $B$ commutes with every $A^k$ and so with $e^{tA}$ .
Let $F(t) = e^{tA}e^{tB}$ . Then $F' = Ae^{tA}e^{tB} + e^{tA}Be^{tB}$ .
$F' = AF + Be^{tA}e^{tB} = (A + B)F$ .
$F(0) = I$ and $e^{t(A+B)}$ solves the same problem, so $F(t) = e^{t(A+B)}$ .

Quick checkFor which pair of matrices is

e^{A+B} = e^Ae^B

guaranteed?

Corollary T MATH 376 practice final 7(b)

e^A is always invertible

For any square $A$ : $e^Ae^{-A} = e^{-A}e^A = I$ , so $e^A$ is invertible with inverse $e^{-A}$ . (Label B: $\det e^A = e^{\operatorname{tr}A}$ , hence $\det e^A > 0$ for a real matrix.)

In one lineFor every square matrix $A$ , $e^A$ is invertible and $(e^A)^{-1} = e^{-A}$ . Moreover $\det e^A = e^{\operatorname{tr}A}$ (a fact beyond the course).

Why it is true $A$ and $-A$ commute, so $e^Ae^{-A} = e^0 = I$ . The determinant formula follows from the triangular form of $A$ .

ProofMATH 376 practice final 7(b); 3.6

The idea: A and -A commute, so the law of exponents applies. For the determinant, use the Schur form A = U T U* and the fact that e^T is triangular with diagonal entries e to the power of the diagonal entries of T.

e^Ae^{-A} = e^{A + (-A)} = e^0 = I

, and in the other order the same.

A(-A) = (-A)A

, so the commuting law applies with

B = -A

e^A

is invertible, with inverse

e^{-A}

A square matrix with a two-sided inverse is invertible (2.4).

Let

A = UTU^*

(B) Schur (3.6):

A = UTU^*

with

T

upper triangular and

U

unitary. Powers conjugate:

A^k = UT^kU^*

, so

e^A = Ue^TU^*

e^T

is upper triangular with diagonal

e^{t_{11}}, \dots, e^{t_{nn}}

Powers of an upper triangular matrix are upper triangular with the diagonal entries raised to the same power, so the diagonal of

e^T

has entries

\sum_kt_{ii}^k/k! = e^{t_{ii}}

\det e^A = \prod e^{t_{ii}} = e^{\sum t_{ii}} = e^{\operatorname{tr}A}

The determinant of a triangular matrix is the product of its diagonal (2.7); the product rule for determinants kills

U

and

U^*

; the diagonal of

T

is the eigenvalues, which add up to the trace.

\blacksquare

Computing e^(tA)

The series is a definition, not a method. We want closed forms. Five cases, in the order you should try them.

Fact T Technau, April 20 (Lemma 8.8); MATH 376 worksheet 25 #1

The exponential of a diagonal matrix

If $D = \operatorname{diag}(d_1, \dots, d_n)$ then $D^k = \operatorname{diag}(d_1^k, \dots, d_n^k)$ and $e^{tD} = \operatorname{diag}(e^{d_1t}, \dots, e^{d_nt})$ . The system $x' = Dx$ is $n$ separate equations $x_i' = d_ix_i$ , with solutions $x_i = c_ie^{d_it}$ .

In one line $e^{t\operatorname{diag}(d_1, \dots, d_n)} = \operatorname{diag}(e^{d_1t}, \dots, e^{d_nt})$ .

Why it is truePowers of a diagonal matrix are diagonal, entry by entry, so the series splits into $n$ ordinary exponential series.

Theorem T Technau, April 20 (Lemma 8.9); MATH 376 worksheet 25 #2; Apostol 7.10

Exponential of a diagonalizable matrix

Let $A = CDC^{-1}$ with $C$ invertible and $D = \operatorname{diag}(\lambda_1, \dots, \lambda_n)$ . Then $e^{tA} = C\operatorname{diag}(e^{\lambda_1t}, \dots, e^{\lambda_nt})C^{-1}$ . The columns of $C$ are eigenvectors of $A$ with eigenvalues $\lambda_1, \dots, \lambda_n$ (in the same order).

In one lineIf $A = CDC^{-1}$ with $D$ diagonal, then $e^{tA} = Ce^{tD}C^{-1}$ .

Why it is truePowers of $A$ are $CD^kC^{-1}$ , and $C$ and $C^{-1}$ factor out of the whole series.

ProofTechnau, April 20

The idea: Powers: A^k = C D^k C^(-1). Sum the series and pull C out of the sum.

A^k = CD^kC^{-1}

Middle factors

C^{-1}C = I

cancel (the quiz of this chapter).

e^{tA} = \sum_k\frac{t^k}{k!}CD^kC^{-1} = C\Big(\sum_k\frac{t^kD^k}{k!}\Big)C^{-1}

Multiplication by the fixed matrices

C

and

C^{-1}

is continuous and linear, so it passes through a convergent series.

= Ce^{tD}C^{-1}

\blacksquare

The diagonal case.

Worked example MATH 376 worksheet 25 #4

Compute e^(tA) and the solution with x(0) = (1, 0) for A = [[3, 1], [1, 3]].

Solution. Use the stepper to follow the six moves.

Answer. $e^{tA} = \frac12\begin{bmatrix}e^{4t} + e^{2t} & e^{4t} - e^{2t}\\e^{4t} - e^{2t} & e^{4t} + e^{2t}\end{bmatrix}$ and $x(t) = \frac12\big(e^{4t} + e^{2t},\ e^{4t} - e^{2t}\big)$ .

Worked example MATH 376 worksheet 25 #5; homework 10 #3

A triangular matrix and a diagonal one: A = [[5, 2], [0, 1]] with x(0) = (a, b); and A = diag(1, 2, ..., n) with y(0) = (1, 2, ..., n).

Solution.

Eigenvalues of the triangular matrix. They are the diagonal entries $5$ and $1$ . The characteristic polynomial of a triangular matrix is the product of the $\lambda - a_{ii}$ .
Eigenvectors. For $5$ : $A - 5I = \begin{bmatrix}0 & 2\\0 & -4\end{bmatrix}$ forces the second coordinate $0$ , so $v_1 = (1, 0)$ . For $1$ : $A - I = \begin{bmatrix}4 & 2\\0 & 0\end{bmatrix}$ gives $2x + y = 0$ , so $v_2 = (1, -2)$ . Eigenvalues on the diagonal, but the eigenvectors still have to be computed.
Assemble. $C = \begin{bmatrix}1 & 1\\0 & -2\end{bmatrix}$ , $C^{-1} = \begin{bmatrix}1 & \frac12\\0 & -\frac12\end{bmatrix}$ , so $e^{tA} = \begin{bmatrix}e^{5t} & e^t\\0 & -2e^t\end{bmatrix}\begin{bmatrix}1 & \frac12\\0 & -\frac12\end{bmatrix} = \begin{bmatrix}e^{5t} & \frac{e^{5t} - e^t}{2}\\0 & e^t\end{bmatrix}$ . Multiply $Ce^{tD}$ by $C^{-1}$ .
The solution. $x(t) = \big(ae^{5t} + b\frac{e^{5t} - e^t}{2},\ be^t\big)$ . Apply to $(a, b)$ .
The diagonal matrix. $\operatorname{diag}(1, \dots, n)$ gives $y_i' = iy_i$ , $y_i(0) = i$ , so $y_i = ie^{it}$ and $y(t) = (e^t, 2e^{2t}, \dots, ne^{nt})$ . Decoupled equations; no matrix to invert.

Answer. $x(t) = \big(ae^{5t} + b\frac{e^{5t} - e^t}{2},\ be^t\big)$ and $y_i(t) = ie^{it}$ .

When the eigenvalues are complex, the formula still holds over $\mathbb C$ , and the imaginary parts cancel in the end. There are two shortcuts that avoid complex arithmetic altogether.

Theorem T MATH 376 worksheet 24 #6(c), worksheet 26 #3; Technau, April 22 (Example 8.11)

Exponential of a rotation-scaling block

For real $a, b$ : $e^{t(aI + bJ)} = e^{at}\big(\cos bt\,I + \sin bt\,J\big) = e^{at}\begin{bmatrix}\cos bt & -\sin bt\\\sin bt & \cos bt\end{bmatrix}$ . The eigenvalues of the block are $a \pm ib$ .

In one lineFor $A = \begin{bmatrix}a & -b\\b & a\end{bmatrix} = aI + bJ$ with $J = \begin{bmatrix}0 & -1\\1 & 0\end{bmatrix}$ : $e^{tA} = e^{at}\begin{bmatrix}\cos bt & -\sin bt\\\sin bt & \cos bt\end{bmatrix}$ .

Why it is true $aI$ commutes with $bJ$ , so the exponential splits as $e^{at}$ times $e^{btJ}$ ; and $J^2 = -I$ turns the series into cosine and sine.

ProofMATH 376 worksheet 26 #3

The idea: Split off the scalar part with the law of exponents. Then J squared is minus the identity, so even powers give cosine and odd powers give sine.

e^{t(aI + bJ)} = e^{atI}e^{btJ} = e^{at}e^{btJ}

aI

commutes with every matrix, so the commuting law applies.

J^2 = -I

Direct multiplication:

J^2 = \begin{bmatrix}-1 & 0\\0 & -1\end{bmatrix}

. So

J^{2m} = (-1)^mI

and

J^{2m+1} = (-1)^mJ

e^{btJ} = \sum_m\frac{(-1)^m(bt)^{2m}}{(2m)!}I + \sum_m\frac{(-1)^m(bt)^{2m+1}}{(2m+1)!}J = \cos bt\,I + \sin bt\,J

\blacksquare

Even powers of

btJ

give multiples of

I

, odd powers multiples of

J

; the coefficients are the series of cosine and sine.

For $a = 0$ , $b = 1$ this says that $e^{tJ}$ is the rotation by the angle $t$ : the solution of $x' = Jx$ is a point going round the origin at unit speed, which is the picture’s “centre”. Technau’s April 22 example gets the same matrix by diagonalizing over $\mathbb C$ : $J$ has eigenvalues $\pm i$ and eigenvectors $(1, \mp i)$ , and the product $C\operatorname{diag}(e^{it}, e^{-it})C^{-1}$ simplifies by Euler’s formula to the rotation. So the answer is real although the working is complex.

Worked example MATH 376 worksheet 26 #4

Complex eigenvalues, block form: x' = [[1, −2], [2, 1]]x with x(0) = (3, −1).

Solution.

Recognise the block. $A = \begin{bmatrix}1 & -2\\2 & 1\end{bmatrix}$ has the form $aI + bJ$ with $a = 1$ , $b = 2$ . Diagonal entries equal, off-diagonal entries opposite.
Write $e^{tA}$ . $e^{tA} = e^t\begin{bmatrix}\cos2t & -\sin2t\\\sin2t & \cos2t\end{bmatrix}$ . The block theorem.
Apply to $(3, -1)$ . $x(t) = e^t\big(3\cos2t + \sin2t,\ 3\sin2t - \cos2t\big)$ . First coordinate $3\cos2t - (-1)\sin2t$ ; second $3\sin2t + (-1)\cos2t$ .
Check. At $t = 0$ this is $(3, -1)$ . Starting value.

Answer. $x(t) = e^t(3\cos2t + \sin2t,\ 3\sin2t - \cos2t)$ : a spiral growing like $e^t$ .

Not every real matrix with complex eigenvalues is of the block form, but every $2 \times 2$ one satisfies a short formula, thanks to the Cayley–Hamilton theorem. This is what the final’s Problem 5(b) needed.

Theorem A Apostol 7.13 (formulas 7.33 to 7.35); MATH 376 final 5(b)

Exponential of a 2 by 2 matrix

Let $A$ be a $2 \times 2$ matrix with eigenvalues $\lambda, \mu$ (repeated if the characteristic polynomial has a double root). Then the three formulas above hold. In the third, $a, b$ are real and $b \ne 0$ whenever $A$ is real with complex eigenvalues. Equivalently in the complex case: $(A - aI)^2 = -b^2I$ .

In one lineFor a $2 \times 2$ matrix with eigenvalues $\lambda, \mu$ : if $\lambda = \mu$ , $e^{tA} = e^{\lambda t}\big(I + t(A - \lambda I)\big)$ ; if $\lambda \ne \mu$ , $e^{tA} = e^{\lambda t}I + \frac{e^{\lambda t} - e^{\mu t}}{\lambda - \mu}(A - \lambda I)$ ; if $\lambda, \mu = a \pm ib$ , $e^{tA} = e^{at}\big(\cos bt\,I + \frac{\sin bt}{b}(A - aI)\big)$ .

Why it is trueBy Cayley–Hamilton $(A - \lambda I)(A - \mu I) = O$ , so every power of $A$ is a combination of $I$ and $A - \lambda I$ , and so is the whole series. The coefficients are found from a first-order system.

ProofApostol 7.13

The idea: This is Putzer's theorem for n = 2, proved in general below. Here we give the complex case directly from (A - aI)^2 = -b^2 I.

(A - aI)^2 + b^2I = O

, that is,

(A - aI)^2 = -b^2I

The characteristic polynomial of

A

(t - a)^2 + b^2

(roots

a \pm ib

). By the Cayley–Hamilton theorem (3.6),

A

satisfies it.

Put

N = A - aI

. Then

e^{tA} = e^{at}e^{tN}

with

N^2 = -b^2I

aI

commutes with

A - aI

, so

e^{tA} = e^{at}e^{t(A - aI)}

by the commuting law.

e^{tN} = \cos bt\,I + \frac{\sin bt}{b}N

N^{2m} = (-b^2)^mI

and

N^{2m+1} = (-b^2)^mN

. Even powers give cosine, odd powers give

\frac1b

times sine, exactly as in the block theorem.

e^{tA} = e^{at}\big(\cos bt\,I + \frac{\sin bt}{b}(A - aI)\big)

Substitute back. The other two cases are the general Putzer formulas with

n = 2

(below).

\blacksquare

Worked example MATH 376 final 5(b)

MATH 376 final 5(b): solve y1' = y1 + 2y2, y2' = −2y1 + 3y2, and give the real general solution.

Solution.

Write the matrix. $A = \begin{bmatrix}1 & 2\\-2 & 3\end{bmatrix}$ and $y' = Ay$ . This is as far as you got on the exam, and it earned $1$ of $4$ .
Plan, in case time runs out. Eigenvalues come from $\lambda^2 - 4\lambda + 7$ , so $\lambda = 2 \pm i\sqrt3$ , and the real solutions are combinations of $e^{2t}\cos\sqrt3t$ and $e^{2t}\sin\sqrt3t$ . The exam cover says that a plan, written in notation, earns partial credit. This line alone would have earned more than the matrix did.
Find the characteristic polynomial. Trace $= 4$ , determinant $= 1\cdot3 - 2\cdot(-2) = 7$ , so $\lambda^2 - 4\lambda + 7 = 0$ and $\lambda = \frac{4 \pm \sqrt{16 - 28}}{2} = 2 \pm i\sqrt3$ . Trace and determinant give the $2 \times 2$ polynomial at once (2.8).
Use the shortcut. With $a = 2$ , $b = \sqrt3$ : $A - 2I = \begin{bmatrix}-1 & 2\\-2 & 1\end{bmatrix}$ , and $(A - 2I)^2 = \begin{bmatrix}1 - 4 & -2 + 2\\2 - 2 & -4 + 1\end{bmatrix} = -3I$ . This confirms $(A - aI)^2 = -b^2I$ with $b^2 = 3$ .
Write the exponential. $e^{tA} = e^{2t}\Big(\cos\sqrt3t\,I + \frac{\sin\sqrt3t}{\sqrt3}(A - 2I)\Big) = e^{2t}\begin{bmatrix}\cos\sqrt3t - \frac{\sin\sqrt3t}{\sqrt3} & \frac{2\sin\sqrt3t}{\sqrt3}\\-\frac{2\sin\sqrt3t}{\sqrt3} & \cos\sqrt3t + \frac{\sin\sqrt3t}{\sqrt3}\end{bmatrix}$ . Substitute $A - 2I$ into the formula.
Apply to $(c_1, c_2)$ with real constants. $y_1 = e^{2t}\Big[c_1\cos\sqrt3t + \frac{2c_2 - c_1}{\sqrt3}\sin\sqrt3t\Big]$ and $y_2 = e^{2t}\Big[c_2\cos\sqrt3t + \frac{c_2 - 2c_1}{\sqrt3}\sin\sqrt3t\Big]$ . The problem asks for real-valued solutions, and $c_1 = y_1(0)$ , $c_2 = y_2(0)$ are real.
Check. $y_1'(0) = 2c_1 + (2c_2 - c_1) = c_1 + 2c_2$ , which equals $y_1(0) + 2y_2(0)$ . The first equation of the system at $t = 0$ .

Answer. The general real solution is the pair $y_1, y_2$ above. The point spirals outward (growth rate $e^{2t}$ , angular speed $\sqrt3$ ), as in the last preset of the picture.

Your answerIn the solution of final 5(b) with

c_1 = 1

c_2 = 0

y_1(t) = e^{2t}\big[\cos\sqrt3t - \frac{1}{\sqrt3}\sin\sqrt3t\big]

. What is

y_1\!\big(\pi/\sqrt3\big)

? (One decimal.)

The $(A - aI)^2 = -b^2I$ shortcut is only for $2 \times 2$ matrices. For a complex eigenvalue of a matrix of any size there is a second route that never leaves real numbers: take the real and imaginary parts of one complex solution.

Theorem T Technau, April 22; MATH 376 course notes (real and imaginary parts); Apostol 7.10

Real and imaginary parts of a complex solution

Let $A$ be a real $n \times n$ matrix and $Av = \lambda v$ with $\lambda = a + ib$ , $b \ne 0$ , $v = u + iw$ ( $u, w \in \mathbb R^n$ ). Then $z(t) = e^{\lambda t}v = x_1(t) + i\,x_2(t)$ , with $x_1, x_2$ real vector functions as above. Both solve $x' = Ax$ , and they are linearly independent. The same holds for the conjugate eigenvalue $\bar\lambda$ and gives nothing new. Together with the solutions from the other eigenvalues they give $n$ independent real solutions.

In one lineIf $A$ is real and $Av = \lambda v$ with $\lambda = a + ib$ , $b \ne 0$ , and $v = u + iw$ with $u, w$ real vectors, then $x_1 = e^{at}(\cos bt\,u - \sin bt\,w)$ and $x_2 = e^{at}(\sin bt\,u + \cos bt\,w)$ are two independent real solutions of $x' = Ax$ .

Why it is trueThe solution $z = e^{\lambda t}v$ is complex, but $A$ is real, so the real part and the imaginary part of the equation $z' = Az$ are two separate real equations. Each part is a real solution.

Proofcourse notes (real and imaginary parts)

The idea: Write the complex solution as real part plus i times imaginary part, and compare real and imaginary parts of z' = Az using that A is real. Independence comes from the starting vectors u and w.

e^{\lambda t}v = e^{at}\big[(\cos bt\,u - \sin bt\,w) + i(\sin bt\,u + \cos bt\,w)\big] = x_1 + ix_2

Euler's formula:

e^{\lambda t} = e^{at}(\cos bt + i\sin bt)

. Multiply out with

v = u + iw

and use

i^2 = -1

z = e^{\lambda t}v

satisfies

z' = Az

The eigenvector theorem above holds for complex

\lambda

and

v

x_1' = Ax_1

and

x_2' = Ax_2

z' = x_1' + ix_2'

and, because

A

has real entries,

Az = Ax_1 + iAx_2

with

Ax_1, Ax_2

real. Two complex vectors are equal exactly when their real parts and their imaginary parts are equal.

\alpha x_1 + \beta x_2 = O

for all

t

, then at

t = 0

we get

\alpha u + \beta w = O

, so

\alpha = \beta = 0

. The solutions are independent.

\blacksquare

t = 0

x_1(0) = u

and

x_2(0) = w

. The vectors

v

and

\bar v

are eigenvectors for the different eigenvalues

\lambda \ne \bar\lambda

, so they are independent over

\mathbb C

(2.8). Now

u = \frac12(v + \bar v)

and

w = \frac1{2i}(v - \bar v)

are obtained from

v, \bar v

by an invertible change, so

u, w

are independent too.

Worked example MATH 376 final 5(b), course notes

MATH 376 final 5(b) a third way: real and imaginary parts of e^(λt)v for A = [[1, 2], [−2, 3]].

Solution.

Eigenvalue. $\lambda = 2 + i\sqrt3$ , from $\lambda^2 - 4\lambda + 7 = 0$ . Trace $4$ , determinant $7$ as before.
Eigenvector. $(A - \lambda I)v = O$ has first row $(-1 - i\sqrt3)v_1 + 2v_2 = 0$ . Take $v_1 = 2$ and $v_2 = 1 + i\sqrt3$ . Check the second row: $-2 \cdot 2 + (1 - i\sqrt3)(1 + i\sqrt3) = -4 + 1 + 3 = 0$ .
Split $v$ . $v = u + iw$ with $u = (2, 1)$ and $w = (0, \sqrt3)$ . $u$ is the real part of $v$ and $w$ the imaginary part.
Write the two real solutions. $x_1 = e^{2t}\big(\cos\sqrt3t\,u - \sin\sqrt3t\,w\big) = e^{2t}\big(2\cos\sqrt3t,\ \cos\sqrt3t - \sqrt3\sin\sqrt3t\big)$ and $x_2 = e^{2t}\big(\sin\sqrt3t\,u + \cos\sqrt3t\,w\big) = e^{2t}\big(2\sin\sqrt3t,\ \sin\sqrt3t + \sqrt3\cos\sqrt3t\big)$ . The theorem, with $a = 2$ , $b = \sqrt3$ .
General real solution. $x = k_1x_1 + k_2x_2$ with real $k_1, k_2$ . Two independent solutions of a system in the plane are a basis of the solution space.
Match the starting vector. $x(0) = (2k_1,\ k_1 + \sqrt3k_2) = (c_1, c_2)$ gives $k_1 = \frac{c_1}{2}$ and $k_2 = \frac{2c_2 - c_1}{2\sqrt3}$ . Then $y_1 = e^{2t}\big[c_1\cos\sqrt3t + \frac{2c_2 - c_1}{\sqrt3}\sin\sqrt3t\big]$ . This is the first coordinate found before with $e^{tA}$ . The second coordinate matches too.

Answer. The same general real solution as before, now found without a matrix exponential. For a matrix with complex eigenvalues of size $3 \times 3$ or more this is the route to use.

Quick checkA real

2 \times 2

matrix has the eigenvalue

1 + 2i

with eigenvector

v = (1, i)

. Which is a real solution of

x' = Ax

Next, the matrices that cannot be diagonalized (such a matrix is called defective). (A matrix can be diagonalized exactly when, for every eigenvalue, the number of independent eigenvectors equals the multiplicity of the eigenvalue as a root of the characteristic polynomial: 2.8.)

Definition T

Nilpotent matrix

A square matrix $N$ is nilpotent if $N^k = O$ for some positive integer $k$ . Example: $N = \begin{bmatrix}0 & 1\\0 & 0\end{bmatrix}$ has $N^2 = O$ .

In one lineA square matrix $N$ with $N^k = O$ for some $k$ .

Why it existsThe exponential series of a nilpotent matrix stops after $k$ terms, so it is a polynomial in $t$ . Every defective matrix is a scalar matrix plus a nilpotent one on each Jordan block.

Theorem T MATH 376 worksheet 24 #6(b), worksheet 28 #5; Apostol 7.14

Exponential of a scalar plus a nilpotent matrix

Let $A = \lambda I + N$ with $N^k = O$ . Then $e^{tA} = e^{\lambda t}\sum_{j=0}^{k-1}\frac{t^jN^j}{j!}$ . In particular, for $N^2 = O$ : $e^{tA} = e^{\lambda t}(I + tN)$ .

In one lineIf $A = \lambda I + N$ with $N^k = O$ , then $e^{tA} = e^{\lambda t}\Big(I + tN + \frac{t^2N^2}{2!} + \dots + \frac{t^{k-1}N^{k-1}}{(k-1)!}\Big)$ .

Why it is true $\lambda I$ commutes with $N$ , so the exponential splits, and the series of $e^{tN}$ ends at $N^{k-1}$ .

ProofMATH 376 worksheet 24 #6(b)

The idea: Law of exponents for the commuting pair λI and N, and the series of e^(tN) is a finite sum.

e^{tA} = e^{\lambda t}e^{tN}

\lambda I

commutes with

N

, so

e^{t(\lambda I + N)} = e^{t\lambda I}e^{tN}

e^{tN} = \sum_{j<k}\frac{t^jN^j}{j!}

N^j = O

for

j \ge k

, so all later terms of the series are zero.

\blacksquare

Worked example MATH 376 worksheet 24 #6(b), worksheet 28 #5; new

Defective matrices: A = [[1, 1], [0, 1]]; x' = [[2, 1], [0, 2]]x with x(0) = (1, 1); and A = [[3, 1], [−1, 1]].

Solution.

Split the first matrix. $A = I + N$ with $N = \begin{bmatrix}0 & 1\\0 & 0\end{bmatrix}$ and $N^2 = O$ . The only eigenvalue is $1$ , doubled, with a single eigendirection, so $A$ cannot be diagonalized.
Apply the theorem. $e^{tA} = e^t(I + tN) = e^t\begin{bmatrix}1 & t\\0 & 1\end{bmatrix}$ . This is the shear of 2.8, now with a growing factor.
Second system. $\begin{bmatrix}2 & 1\\0 & 2\end{bmatrix} = 2I + N$ , so $e^{tA} = e^{2t}\begin{bmatrix}1 & t\\0 & 1\end{bmatrix}$ and $x(t) = e^{2t}\begin{bmatrix}1 + t\\1\end{bmatrix}$ . Apply to $(1, 1)$ : first coordinate $1\cdot1 + t\cdot1$ .
Check. $x_2' = 2x_2$ since $x_2 = e^{2t}$ . And $x_1' = (3 + 2t)e^{2t} = 2x_1 + x_2 = 2(1 + t)e^{2t} + e^{2t}$ . Both equations of the system hold.
The third matrix needs no eigenvectors. The trace is $4$ and the determinant is $3 + 1 = 4$ , so the characteristic polynomial is $(\lambda - 2)^2$ . Then $A - 2I = \begin{bmatrix}1 & 1\\-1 & -1\end{bmatrix}$ and $(A - 2I)^2 = O$ (by Cayley–Hamilton, or by multiplying). So $e^{tA} = e^{2t}\big(I + t(A - 2I)\big) = e^{2t}\begin{bmatrix}1 + t & t\\-t & 1 - t\end{bmatrix}$ . For a $2 \times 2$ matrix with a double eigenvalue $\lambda$ , $A - \lambda I$ is nilpotent, so this works without computing a Jordan basis.

Answer. $e^t\begin{bmatrix}1 & t\\0 & 1\end{bmatrix}$ ; $x(t) = e^{2t}(1 + t, 1)$ ; $e^{2t}\begin{bmatrix}1 + t & t\\-t & 1 - t\end{bmatrix}$ .

Your answerFor

x' = \begin{bmatrix}2 & 1\\0 & 2\end{bmatrix}x

x(0) = (1, 1)

, the solution is

x(t) = e^{2t}(1 + t, 1)

. What is the first coordinate at

t = 1

? (Three decimals.)

For larger matrices the general statement is the Jordan form. If $A = CJC^{-1}$ (3.6) then, exactly as in the diagonalizable theorem, $e^{tA} = Ce^{tJ}C^{-1}$ , and $e^{tJ}$ is computed block by block: for a Jordan block $J_k(\lambda) = \lambda I + N$ with $N$ the matrix with ones directly above the diagonal, $N^k = O$ and the nilpotent theorem applies. For example,

e^{t\begin{bmatrix}2 & 1 & 0\\0 & 2 & 1\\0 & 0 & 2\end{bmatrix}} = e^{2t}\begin{bmatrix}1 & t & \frac{t^2}{2}\\0 & 1 & t\\0 & 0 & 1\end{bmatrix}.

This is the matrix reason why a root of multiplicity $m$ in 7.2 gives $e^{rt}, te^{rt}, \dots, t^{m-1}e^{rt}$ : a Jordan block of size $m$ has $N^m = O$ and $e^{tN}$ contains the powers up to $t^{m-1}$ .

Theorem A Apostol 7.13 (Theorem 7.9); Putzer 1966

Putzer's method

Let $A$ be $n \times n$ with eigenvalues $\lambda_1, \dots, \lambda_n$ (in any order, repeated according to multiplicity). Define $P_0 = I$ and $P_k = \prod_{m=1}^{k}(A - \lambda_mI)$ . Let $r_1, \dots, r_n$ be the functions determined by the triangular system $r_1' = \lambda_1r_1$ , $r_1(0) = 1$ ; $r_{k+1}' = \lambda_{k+1}r_{k+1} + r_k$ , $r_{k+1}(0) = 0$ ( $1 \le k < n$ ). Then $e^{tA} = r_1(t)P_0 + r_2(t)P_1 + \dots + r_n(t)P_{n-1}$ .

In one lineWith eigenvalues $\lambda_1, \dots, \lambda_n$ listed with multiplicity and $P_0 = I$ , $P_k = (A - \lambda_1I)\cdots(A - \lambda_kI)$ , one has $e^{tA} = \sum_{k=0}^{n-1}r_{k+1}(t)P_k$ , where $r_1' = \lambda_1r_1$ , $r_1(0) = 1$ and $r_{k+1}' = \lambda_{k+1}r_{k+1} + r_k$ , $r_{k+1}(0) = 0$ .

Why it is trueThe right-hand side $F(t)$ satisfies $F' = AF$ and $F(0) = I$ thanks to the Cayley–Hamilton theorem, so it equals $e^{tA}$ by uniqueness. No eigenvectors and no Jordan form are needed.

ProofApostol 7.13

The idea: Define F as the right-hand side. Show F' = AF using the recursion for the r_k and the relation P_{k+1} = (A - λ_{k+1}I)P_k, with Cayley–Hamilton killing the last term. F(0) = I, so F = e^(tA) by uniqueness.

Let

F(t) = \sum_{k=0}^{n-1}r_{k+1}(t)P_k

. Then

F(0) = I

t = 0

only

r_1

is non-zero:

r_1(0) = 1

r_k(0) = 0

for

k > 1

, and

P_0 = I

F' = \sum_{k=0}^{n-1}\big(\lambda_{k+1}r_{k+1} + r_k\big)P_k

The recursion, with the convention

r_0 = 0

F' - \lambda_nF = \sum_{k=0}^{n-2}r_{k+1}\big[(\lambda_{k+1} - \lambda_n)P_k + P_{k+1}\big]

Subtract

\lambda_nF

. The

k = n - 1

term of

\sum(\lambda_{k+1} - \lambda_n)r_{k+1}P_k

is zero, and the sum

\sum_kr_kP_k

is re-indexed by

k \to k + 1

F' - \lambda_nF = (A - \lambda_nI)\sum_{k=0}^{n-2}r_{k+1}P_k = (A - \lambda_nI)\big(F - r_nP_{n-1}\big)

P_{k+1} = (A - \lambda_{k+1}I)P_k

, so the bracket is

(A - \lambda_{k+1}I + \lambda_{k+1}I - \lambda_nI)P_k = (A - \lambda_nI)P_k

F' - \lambda_nF = (A - \lambda_nI)F - r_nP_n = (A - \lambda_nI)F

(A - \lambda_nI)P_{n-1} = P_n = p_A(A)

, where

p_A

is the characteristic polynomial, and

p_A(A) = O

by the Cayley–Hamilton theorem (3.6).

F' = AF

Cancel

\lambda_nF

on both sides.

F(t) = e^{tA}

e^{tA}

solves the same IVP

X' = AX

X(0) = I

, and the solution is unique (7.1).

\blacksquare

Worked example Apostol 7.13, Examples 1 and 2; MATH 376 worksheet 25 #5

Putzer's method for [[5, 2], [0, 1]], and the general repeated case.

Solution.

Eigenvalues, in any order. $\lambda_1 = 5$ , $\lambda_2 = 1$ . Triangular matrix: the diagonal.
The matrices $P_k$ . $P_0 = I$ and $P_1 = A - 5I = \begin{bmatrix}0 & 2\\0 & -4\end{bmatrix}$ . $P_k$ is the product of the first $k$ factors $(A - \lambda_mI)$ .
Solve the system for $r_1, r_2$ . $r_1' = 5r_1$ , $r_1(0) = 1$ gives $r_1 = e^{5t}$ . Then $r_2' = r_2 + e^{5t}$ , $r_2(0) = 0$ gives $r_2 = \frac{e^{5t} - e^t}{4}$ . Solve in order: each equation uses the previous $r$ . Check: $r_2' = \frac{5e^{5t} - e^t}{4}$ and $r_2 + e^{5t} = \frac{e^{5t} - e^t + 4e^{5t}}{4}$ .
Assemble. $e^{tA} = e^{5t}I + \frac{e^{5t} - e^t}{4}\begin{bmatrix}0 & 2\\0 & -4\end{bmatrix} = \begin{bmatrix}e^{5t} & \frac{e^{5t} - e^t}{2}\\0 & e^t\end{bmatrix}$ . The same matrix as by diagonalization in the previous example.
The general formulas for $n = 2$ . If $\lambda_1 = \lambda_2 = \lambda$ the system is $r_1' = \lambda r_1$ , $r_2' = \lambda r_2 + r_1$ with solutions $r_1 = e^{\lambda t}$ , $r_2 = te^{\lambda t}$ , so $e^{tA} = e^{\lambda t}I + te^{\lambda t}(A - \lambda I)$ . For $\lambda_1 = \lambda \ne \mu = \lambda_2$ we get $r_2 = \frac{e^{\lambda t} - e^{\mu t}}{\lambda - \mu}$ . These are the formulas of the $2 \times 2$ theorem; with $\lambda = a + ib$ , $\mu = a - ib$ they give the $\cos$ and $\sin$ form.

Answer. $e^{tA} = \begin{bmatrix}e^{5t} & \frac{e^{5t} - e^t}{2}\\0 & e^t\end{bmatrix}$ .

Why· Which method for which matrix

Look at $A$ and ask, in this order. (1) Diagonal? $e^{tA} = \operatorname{diag}(e^{d_it})$ . (2) Scalar plus nilpotent, such as $\begin{bmatrix}2 & 1\\0 & 2\end{bmatrix}$ or any $2 \times 2$ with a double eigenvalue? $e^{\lambda t}(I + tN + \cdots)$ . (3) A $2 \times 2$ with complex eigenvalues $a \pm ib$ ? $e^{at}(\cos bt\,I + \frac{\sin bt}{b}(A - aI))$ , real from the start. (4) Distinct real eigenvalues, any size: diagonalize, $Ce^{tD}C^{-1}$ , or Putzer. (5) Anything else: Jordan form or Putzer. Always end by checking $e^{0A} = I$ and one entry of the differential equation. For a problem that only asks for the SOLUTION with a given starting vector, the eigenvalue method ( $x = \sum c_je^{\lambda_jt}v_j$ ) avoids computing $C^{-1}$ .

Inhomogeneous systems

Theorem T Technau, April 20 (Theorem 8.7); MATH 376 worksheet 26 #1; Apostol 7.16

Variation of constants for systems

Let $A$ be constant and $g$ continuous on an interval $J \ni a$ . The IVP $x' = Ax + g$ , $x(a) = b$ has exactly one solution on $J$ :

x(t) = e^{(t-a)A}b + \int_a^{t}e^{(t-s)A}g(s)\,ds .

In one lineThe IVP $x' = Ax + g(t)$ , $x(a) = b$ has the unique solution $x(t) = e^{(t-a)A}b + \int_a^te^{(t-s)A}g(s)\,ds$ .

Why it is trueSet $x = e^{tA}u$ and let the constant vector $u$ vary. Then $e^{tA}u' = g$ , and $u' = e^{-tA}g$ is integrated at once. The matrix $e^{tA}$ plays the role of the whole homogeneous basis.

ProofMATH 376 worksheet 26 #1

The idea: Make the ansatz x = e^(tA) u(t), derive u' = e^(-tA) g, integrate, and use the law of exponents to combine. Uniqueness is 7.1.

Put

x(t) = e^{tA}u(t)

This is Lagrange's idea of 7.3 for systems: the homogeneous solution

e^{tA}c

with

c

replaced by a function

u(t)

x' = Ae^{tA}u + e^{tA}u' = Ax + e^{tA}u'

Product rule and the derivative theorem.

e^{tA}u' = g

, and

u' = e^{-tA}g

Compare with the equation

x' = Ax + g

u(t) = e^{-aA}b + \int_a^{t}e^{-sA}g(s)\,ds

Integrate from

a

t

, with

u(a) = e^{-aA}b

so that

x(a) = b

x(t) = e^{(t-a)A}b + \int_a^{t}e^{(t-s)A}g(s)\,ds

Multiply by

e^{tA}

and use the law of exponents

e^{tA}e^{-aA} = e^{(t-a)A}

and

e^{tA}e^{-sA} = e^{(t-s)A}

\blacksquare

Worked example MATH 376 worksheet 26 #2

Solve x1' = 2x1 + e^t, x2' = −x2 + t with x(0) = (1, 2).

Solution.

Write $A$ , $g$ . $A = \operatorname{diag}(2, -1)$ , $g(s) = (e^s, s)$ . The equations are decoupled, but the formula works the same way for any $A$ .
Compute $e^{tA}$ . $e^{tA} = \operatorname{diag}(e^{2t}, e^{-t})$ . Diagonal.
The homogeneous part. $e^{tA}x(0) = (e^{2t}, 2e^{-t})$ . Do not forget this term.
The integrand. $e^{(t-s)A}g(s) = \big(e^{2(t-s)}e^s,\ e^{-(t-s)}s\big) = \big(e^{2t-s},\ se^{s-t}\big)$ . Replace $t$ by $t - s$ in $e^{tA}$ and multiply by $g(s)$ . The variable of integration is $s$ .
Integrate each coordinate in $s$ from $0$ to $t$ . First: $e^{2t}\int_0^te^{-s}ds = e^{2t}(1 - e^{-t}) = e^{2t} - e^t$ . Second: $e^{-t}\int_0^tse^sds = e^{-t}(te^t - e^t + 1) = t - 1 + e^{-t}$ . The second uses integration by parts: $\int se^s\,ds = se^s - e^s$ .
Add. $x_1 = e^{2t} + e^{2t} - e^t = 2e^{2t} - e^t$ and $x_2 = 2e^{-t} + t - 1 + e^{-t} = t - 1 + 3e^{-t}$ . Homogeneous part plus the integral.
Check. $x(0) = (2 - 1, -1 + 3) = (1, 2)$ . And $x_1' = 4e^{2t} - e^t = 2x_1 + e^t$ ; $x_2' = 1 - 3e^{-t} = -x_2 + t$ . Starting value and both equations.

Answer. $x(t) = \big(2e^{2t} - e^t,\ t - 1 + 3e^{-t}\big)$ .

The scalar variation of parameters of 7.3 is this formula in disguise: for $y'' + a_1y' + a_0y = R$ the system has $g = (0, R)$ , the matrix $e^{(t-s)A}$ has the top-right entry $G(t, s)$ (the Green’s function), and the first coordinate of the formula is $y_p(t) = \int_a^tG(t, s)R(s)\,ds$ .

Power series and the Neumann series

A third way to solve $y' = ry$ is to assume a power series and match coefficients. It is the method of Technau’s April 22 lecture and worksheet 26.

Worked example Technau, April 22 (Example 8.12); MATH 376 worksheets 26 #5, #6 and 28 #7

Power series: y' = ry; y'' − xy' − y = 0 with y(0) = 1, y'(0) = 0; y'' + y = 0 with y(0) = 0, y'(0) = 1.

Solution.

Assume a series. For $y' = ry$ put $y = \sum_{k\ge0}\alpha_kt^k$ . Then $y' = \sum_{k\ge0}(k + 1)\alpha_{k+1}t^k$ . Differentiate term by term inside the radius of convergence (1.6) and shift the index so that every sum shows $t^k$ .
Match coefficients. $(k + 1)\alpha_{k+1} = r\alpha_k$ , so $\alpha_k = \frac{r^k}{k!}\alpha_0$ , and $y = \alpha_0e^{rt}$ . Two power series are equal when their coefficients agree. Induction on the recurrence.
Second problem: series. Put $y = \sum a_nx^n$ . Then $y'' = \sum(n + 2)(n + 1)a_{n+2}x^n$ and $xy' = \sum_{n\ge1}na_nx^n$ . Every sum is written with $x^n$ .
Recurrence. The $x^0$ coefficient: $2a_2 - a_0 = 0$ . For $n \ge 1$ : $(n + 2)(n + 1)a_{n+2} - na_n - a_n = 0$ , so $a_{n+2} = \frac{a_n}{n + 2}$ , which also covers $n = 0$ . Divide by $n + 1 \ne 0$ .
Starting values. $a_0 = y(0) = 1$ , $a_1 = y'(0) = 0$ . All odd coefficients are $0$ , and $a_2 = \frac12$ , $a_4 = \frac18$ , $a_6 = \frac1{48}$ ; in general $a_{2m} = \frac{1}{2^mm!}$ . $a_{2m} = a_{2m-2}/(2m)$ .
Sum. $y = \sum_m\frac{(x^2/2)^m}{m!} = e^{x^2/2}$ . Check: $y' = xy$ , $y'' = y + xy' = y + x^2y$ , so $y'' - xy' - y = y + x^2y - x^2y - y = 0$ .
Third problem. $a_{n+2} = -\frac{a_n}{(n+2)(n+1)}$ with $a_0 = 0$ , $a_1 = 1$ gives $a_{2m+1} = \frac{(-1)^m}{(2m+1)!}$ , so $y = \sin t$ . The series of sine, recovered from the equation.

Answer. $\alpha_0e^{rt}$ ; $e^{x^2/2}$ ; $\sin t$ .

Theorem T MATH 376 practice final 7(f); Technau, April 15

The Neumann series

Let $A$ be a square matrix with $\lVert A\rVert = \sum_{i,j}\lvert a_{ij}\rvert < 1$ . Then $I - A$ is invertible and $(I - A)^{-1} = I + A + A^2 + \cdots$ .

In one lineIf $\lVert A\rVert < 1$ for the matrix norm of this chapter, then $I - A$ is invertible and $(I - A)^{-1} = \sum_{k\ge0}A^k$ .

Why it is trueThe geometric series of matrices converges because $\lVert A^k\rVert \le \lVert A\rVert^k$ is a geometric series of numbers, and the telescoping sum shows the sum is the inverse.

ProofMATH 376 practice final 7(f)

The idea: Absolute convergence of the matrix geometric series by submultiplicativity, then multiply by I - A and telescope.

\sum_k\lVert A^k\rVert < \infty

Submultiplicativity:

\lVert A^k\rVert \le \lVert A\rVert^k

. The geometric series

\sum\lVert A\rVert^k

converges since

\lVert A\rVert < 1

(1.6).

S = \sum_{k\ge0}A^k

exists.

Absolutely convergent matrix series converge (this chapter).

(I - A)S = \lim_{N}(I - A^{N+1}) = I

Multiplication by

I - A

is continuous and linear, so it passes through the series. The terms telescope:

\sum_{k\le N}A^k - \sum_{k\le N}A^{k+1} = I - A^{N+1}

, and

\lVert A^{N+1}\rVert \le \lVert A\rVert^{N+1} \to 0

Likewise

S(I - A) = I

, so

S = (I - A)^{-1}

The same computation with

S

on the left:

A

commutes with

S

. A matrix with a two-sided inverse is invertible.

\blacksquare

This is the matrix version of the geometric series, and the reason that the iteration $x \mapsto Ax + b$ of 4.2 converges to $(I - A)^{-1}b$ when $\lVert A\rVert < 1$ : the iterates are $b + Ab + A^2b + \cdots$ .

Phase portraits: a first look

We now explain the picture. Everything follows from the formulas for $e^{tA}$ .

Theorem B standard; preview of 7.6

Classification of planar linear systems

Let $A$ be a real $2 \times 2$ matrix with trace $\tau$ and determinant $\Delta \neq 0$ ; its eigenvalues are the roots of $\lambda^2 - \tau\lambda + \Delta$ . (a) $\Delta < 0$ : two real eigenvalues of opposite signs, a saddle. (b) $\Delta > 0$ and $\tau^2 - 4\Delta > 0$ : two real eigenvalues of the same sign, a node, stable if $\tau < 0$ (everything tends to $O$ ) and unstable if $\tau > 0$ . (c) $\Delta > 0$ and $\tau^2 - 4\Delta < 0$ : complex eigenvalues $a \pm ib$ with $a = \tau/2$ , a spiral, stable if $\tau < 0$ , unstable if $\tau > 0$ ; if $\tau = 0$ the orbits are closed curves, a centre. (d) $\tau^2 = 4\Delta$ : a repeated eigenvalue, a degenerate node (straight-line orbits if $A$ is a multiple of $I$ , bent curves round the one eigendirection if not). Every solution tends to $O$ as $t \to \infty$ if and only if both eigenvalues have negative real part, that is $\tau < 0 < \Delta$ .

In one lineFor a real $2 \times 2$ matrix with trace $\tau$ and determinant $\Delta \ne 0$ : $\Delta < 0$ is a saddle; $\Delta > 0$ , $\tau^2 > 4\Delta$ is a node; $\Delta > 0$ , $\tau^2 < 4\Delta$ , $\tau \ne 0$ is a spiral; $\tau = 0 < \Delta$ is a centre. The origin attracts everything exactly when $\tau < 0 < \Delta$ .

Why it is trueThe eigenvalues are the roots of $\lambda^2 - \tau\lambda + \Delta$ . Their signs and whether they are real decide the solutions $c_1e^{\lambda_1t}v_1 + c_2e^{\lambda_2t}v_2$ or $e^{at}$ times a rotation.

Proofstandard

The idea: Read the answer off the formulas for e^(tA) in each case.

(a) The product of the eigenvalues is

\Delta

; negative means opposite signs.

Real distinct eigenvalues:

x = c_1e^{\lambda_1t}v_1 + c_2e^{\lambda_2t}v_2

by the eigenvector theorem. If

\lambda_1 > 0 > \lambda_2

the

v_1

-part grows and the

v_2

-part decays: orbits approach the

v_2

line and then run off along

v_1

, a saddle. Their product is

\Delta

, so (a) is

\Delta < 0

(b) Same sign: all orbits tend to

O

(or all run off), according to the sign of

\tau

Both

e^{\lambda_it}

tend to

0

if both

\lambda_i < 0

, and both blow up if both are positive. The sum is

\tau

O

while their distance changes by

e^{at}

By the

2\times2

theorem

e^{tA} = e^{at}\big(\cos bt\,I + \frac{\sin bt}{b}(A - aI)\big)

: a bounded oscillating factor times

e^{at}

. The factor

e^{at}

shrinks or grows according to the sign of

a = \tau/2

;

a = 0

gives a periodic motion.

(d) Repeated eigenvalue: the nilpotent theorem.

A repeated eigenvalue gives

e^{\lambda t}(I + tN)

with

N^2 = O

; the decay or growth is that of

e^{\lambda t}

with

\lambda = \tau/2

, and the term

tN

bends the orbits when

N \ne O

So all orbits tend to

O

exactly when

\tau < 0 < \Delta

In (b), (c), (d) the long-term behaviour is governed by

\operatorname{Re}\lambda

, and

\tau = \lambda_1 + \lambda_2

\Delta = \lambda_1\lambda_2

. For real roots, both negative is

\tau < 0 < \Delta

; for complex roots the real part

\tau/2 < 0

with

\Delta > 0

\blacksquare

Worked example MATH 376 final 5(b), worksheets 24 to 26 and 28; new

Classify the origin for: [[1, 2], [2, −1]]; [[1, 2], [−2, 3]]; [[3, 1], [1, 3]]; [[5, 2], [0, 1]]; [[2, 1], [0, 2]]; [[0, −1], [1, 0]]; [[−3, 1], [0, −1]].

Solution.

Compute $\tau$ , $\Delta$ and $\tau^2 - 4\Delta$ for each. The table below lists them. Trace and determinant are the only inputs.

Matrix	$\tau$	$\Delta$	$\tau^2 - 4\Delta$	Eigenvalues	Type
$\begin{bmatrix}1 & 2\\2 & -1\end{bmatrix}$	$0$	$-5$	$20$	$\pm\sqrt5$	saddle
$\begin{bmatrix}1 & 2\\-2 & 3\end{bmatrix}$	$4$	$7$	$-12$	$2 \pm i\sqrt3$	unstable spiral
$\begin{bmatrix}3 & 1\\1 & 3\end{bmatrix}$	$6$	$8$	$4$	$4, 2$	unstable node
$\begin{bmatrix}5 & 2\\0 & 1\end{bmatrix}$	$6$	$5$	$16$	$5, 1$	unstable node
$\begin{bmatrix}2 & 1\\0 & 2\end{bmatrix}$	$4$	$4$	$0$	$2, 2$	unstable degenerate node
$\begin{bmatrix}0 & -1\\1 & 0\end{bmatrix}$	$0$	$1$	$-4$	$\pm i$	centre
$\begin{bmatrix}-3 & 1\\0 & -1\end{bmatrix}$	$-4$	$3$	$4$	$-3, -1$	stable node

Apply the classification. The determinant sign separates the saddle; the discriminant $\tau^2 - 4\Delta$ separates node from spiral; the sign of $\tau$ separates stable from unstable. Each row is a single read-off.

Answer. As in the last column. Only the last matrix has all solutions tending to $O$ . The final’s 5(b) matrix (row 2) spirals outward, with growth rate $e^{2t}$ since the real part of the eigenvalues is $\tau/2 = 2$ .

Quick checkA

2 \times 2

real matrix has trace

-2

and determinant

5

. What do the solutions of

x' = Ax

do?

Quick checkTwo different solutions of

x' = Ax

pass through the same point of the plane at different times. What follows?

Python labComputing e^(tA) two ways

import numpy as np

def expm_series(A, t, terms):
    # e^(tA) = I + tA + (tA)^2/2! + ...  (each new term = previous term times tA / k)
    S, term = np.eye(len(A)), np.eye(len(A))
    for k in range(1, terms):
        term = term @ (t * A) / k
        S = S + term
    return S

def expm_eig(A, t):
    # e^(tA) = C e^(tD) C^(-1), valid when A has n independent eigenvectors
    lam, C = np.linalg.eig(A.astype(complex))
    return (C @ np.diag(np.exp(lam * t)) @ np.linalg.inv(C)).real

A = np.array([[1.0, 2.0], [-2.0, 3.0]])   # final 5(b): eigenvalues 2 +- i sqrt(3)
t = 1.0
ref = expm_eig(A, t)
print("series terms   largest entry error vs eigen method")
for n in [2, 4, 8, 12, 16, 20, 25]:
    print(f"{n:6d}         {np.max(np.abs(expm_series(A, t, n) - ref)):.2e}")

# the same code on a matrix that cannot be diagonalized
S = np.array([[1.0, 1.0], [0.0, 1.0]])     # a shear: e^(tS) = e^t [[1, t], [0, 1]]
exact = np.exp(t) * np.array([[1, t], [0, 1]])
print("\nshear, series with 25 terms, error:", np.max(np.abs(expm_series(S, t, 25) - exact)))
lam, C = np.linalg.eig(S)
print("shear, eigenvector matrix C:\n", C, "\n det C =", np.linalg.det(C))
print("shear, eigen method error:", np.max(np.abs(expm_eig(S, t) - exact)))

Try this. Run it: the series reaches machine precision ( $4 \times 10^{-15}$ ) with 25 terms, and the eigen method $C\,e^{tD}C^{-1}$ gives the same matrix. Both handle the complex eigenvalues of the final's 5(b) matrix. Then the shear $\begin{bmatrix}1 & 1\\0 & 1\end{bmatrix}$ , which cannot be diagonalized: the series is still correct (error $4 \times 10^{-16}$ ), but the matrix of eigenvectors $C$ has determinant about $2 \times 10^{-16}$ (its two columns are the same direction) and the eigen method returns an answer wrong by $2.72 = e$ . Now put a small number eps in the bottom-left corner, S = np.array([[1.0, 1.0], [eps, 1.0]]): the matrix is diagonalizable again, but its eigenvectors are almost parallel, and for eps equal to 1e-2, 1e-6, 1e-10 the eigen method's error grows from about $10^{-15}$ to $10^{-13}$ to $10^{-11}$ (compared with a 40-term series), while the series stays accurate. A defective matrix is the limit of an ill-conditioned eigenvector basis (chapter 3.5 on conditioning).

Where it is used

Used inside maths· Inside maths: one-parameter groups, rotations and the determinant

The family $t \mapsto e^{tA}$ is a one-parameter group: $e^{(s+t)A} = e^{sA}e^{tA}$ and $e^{0A} = I$ . For $A = J$ it is the group of rotations of the plane, $t \mapsto$ rotation by the angle $t$ , and the matrix $J$ (the “infinitesimal rotation”) is its derivative at $t = 0$ . In general, matrices $A$ with $A^{\mathsf T} = -A$ exponentiate to orthogonal matrices (3.3): rotations of $\mathbb R^3$ are $e^{A}$ for a skew-symmetric $A$ , which is how robotics and graphics store orientations. The identity $\det e^{A} = e^{\operatorname{tr}A}$ says that the flow of $x' = Ax$ multiplies areas (volumes) by $e^{t\operatorname{tr}A}$ , and the Abel formula of 7.3 is its scalar case. A continuous-time Markov chain is $p' = Qp$ with $Q$ a rate matrix (columns summing to $0$ , off-diagonal entries non-negative): for two states with rates $a$ and $b$ , $Q = \begin{bmatrix}-a & b\\a & -b\end{bmatrix}$ , the null vector $(b, a)/(a + b)$ is the stationary distribution, and $e^{tQ}$ is a stochastic matrix for every $t \ge 0$ (2.11).

Used in the real world· Medicine: how a drug leaves the blood (two compartments)

A drug is injected into the blood (compartment 1). It is eliminated at rate $0.5$ per hour, passes into tissue (compartment 2) at rate $0.3$ per hour, and returns from tissue at rate $0.2$ per hour. With $x = (x_1, x_2)$ the amounts: $x_1' = -(0.5 + 0.3)x_1 + 0.2x_2$ , $x_2' = 0.3x_1 - 0.2x_2$ , so $A = \begin{bmatrix}-0.8 & 0.2\\0.3 & -0.2\end{bmatrix}$ with trace $-1$ and determinant $0.1$ . The eigenvalues are $-0.887$ and $-0.113$ per hour: a stable node with two time scales. The fast one has half-life $\ln2/0.887 = 0.78$ hours (the drug spreads from blood to tissue) and the slow one $\ln2/0.113 = 6.15$ hours (the drug leaves slowly from the tissue). A graph of blood concentration against time on a log scale therefore shows two straight lines, a steep one and then a shallow one: the two-phase decline that doctors observe for many drugs. The matrix exponential $e^{tA}$ gives the amounts at any time, and the variation of constants formula handles a drip: with infusion rate $g = (g_0, 0)$ the amounts settle at the equilibrium $-A^{-1}g$ .

Used in the real world· Epidemiology: why 'R0 above 1' decides an epidemic (linearised SIR)

In the model of Kermack and McKendrick (1927) a population of $N$ is divided into susceptible $S$ , infected $I$ and recovered $R$ : $S' = -\beta SI/N$ , $I' = \beta SI/N - \gamma I$ , $R' = \gamma I$ . At the start almost everyone is susceptible, $S \approx N$ , so $I' \approx (\beta - \gamma)I$ : a linear equation with solution $I = I_0e^{(\beta - \gamma)t}$ . With $\beta = 0.3$ per day and $\gamma = 0.1$ per day, the growth rate is $0.2$ per day and the number of cases doubles every $\ln2/0.2 = 3.47$ days. The ratio $R_0 = \beta/\gamma = 3$ is the number of people one case infects. If a fraction $p$ of the population is immune, $S \approx (1 - p)N$ and the growth rate becomes $\beta(1 - p) - \gamma$ , which is negative exactly when $p > 1 - \frac1{R_0} = \frac23$ . With $p = 0.6$ the growth rate is still $0.3 \times 0.4 - 0.1 = 0.02$ per day (slow growth), and above $p = \frac23$ the epidemic dies out. The sign of one eigenvalue, the number $\beta(1 - p) - \gamma$ , is the whole question; this is the linearisation at the disease-free state that 7.6 treats in general.

Used in the real world· Machine learning: stability of a linear recurrent layer

A linear recurrent layer with hidden state $h_{k+1} = Wh_k$ has $h_k = W^kh_0$ , and the continuous-time version $h' = Ah$ has $h(t) = e^{tA}h(0)$ . Both fade away exactly when the eigenvalues of $W$ are inside the unit circle (respectively those of $A$ have negative real part). A network with an eigenvalue of $W$ of size $1.1$ multiplies the signal by $1.1^{100} \approx 13780$ over a hundred steps, which is the exploding gradient problem; size $0.9$ gives $0.9^{100} \approx 2.7\times10^{-5}$ , the vanishing gradient. The classification of this chapter (spiral if the eigenvalues are complex, node if real) tells you which of the two behaviours you will see.

Traps

Watch out· Write the plan when the computation does not finish

On the 376 final (5(b)) you wrote the matrix $A = \begin{bmatrix}1 & 2\\-2 & 3\end{bmatrix}$ and stopped: no eigenvalues, no exponential, no real solution. The rubric gave $1$ of $4$ . The cover page of every exam says that a method written in notation earns partial credit. One line, ‘eigenvalues $2 \pm i\sqrt3$ , so $y = e^{2t}(\ldots\cos\sqrt3t + \ldots\sin\sqrt3t)$ ’, would have earned more than the matrix did. The fastest complete route for any $2 \times 2$ with complex eigenvalues is the $(A - aI)^2 = -b^2I$ shortcut above.

Watch out· e^(A+B) is not e^A e^B unless A and B commute

Homework 10 #4 gives the explicit counterexample: $e^Ae^B = \begin{bmatrix}e & e\\0 & 1\end{bmatrix}$ but $e^{A+B} = \begin{bmatrix}e & e - 1\\0 & 1\end{bmatrix}$ . Being invertible, or being diagonalizable, does not rescue it: for $A = \operatorname{diag}(2, 1)$ and $B = \begin{bmatrix}1 & 1\\0 & 2\end{bmatrix}$ (both diagonalizable) one finds $e^{A+B} = e^3\begin{bmatrix}1 & 1\\0 & 1\end{bmatrix}$ but $e^Ae^B = e^3\begin{bmatrix}1 & e - 1\\0 & 1\end{bmatrix}$ . In every use of the law of exponents ask: do they commute? $aI$ and anything commute; $A$ with itself, with $2A$ , with $A^2$ , with $e^A$ , with $A - \lambda I$ commute. That is why $e^{t(\lambda I + N)} = e^{\lambda t}e^{tN}$ is allowed.

Watch out· Do not exponentiate entry by entry

$e^{tA} \neq \big[e^{ta_{ij}}\big]$ . For $A = \begin{bmatrix}0 & 1\\0 & 0\end{bmatrix}$ the entrywise exponentials would give $\begin{bmatrix}1 & e^t\\1 & 1\end{bmatrix}$ but the true $e^{tA} = \begin{bmatrix}1 & t\\0 & 1\end{bmatrix}$ . Only diagonal matrices exponentiate entrywise.

Watch out· Order and matching in C e^(tD) C^(-1)

The factor $C$ goes on the left and $C^{-1}$ on the right. The $j$ -th eigenvalue on the diagonal of $D$ must correspond to the $j$ -th column of $C$ . Check: $e^{0\cdot A}$ must come out as $I$ . And $C$ needs $n$ independent eigenvectors; check $\det C \ne 0$ , or a defective matrix (such as $\begin{bmatrix}2 & 1\\0 & 2\end{bmatrix}$ ) will give nonsense.

Watch out· A defective matrix has polynomial factors as well as exponentials

$\begin{bmatrix}1 & 1\\0 & 1\end{bmatrix}$ has the single eigenvalue $1$ , but $e^{tA} = e^t\begin{bmatrix}1 & t\\0 & 1\end{bmatrix}$ is not $e^tI$ . The $t$ in the corner comes from the nilpotent part. If a $2 \times 2$ matrix has a repeated eigenvalue and is not a multiple of $I$ , use $e^{\lambda t}(I + t(A - \lambda I))$ .

Watch out· Give a real answer when the problem is real

The final asked for ‘general real-valued solutions $y_1, y_2 : \mathbb R \to \mathbb R$ ’. An answer with $e^{(2+i\sqrt3)t}$ and complex constants is not what was asked. Use the real shortcut, or take real and imaginary parts, and keep $c_1, c_2$ real.

Watch out· Variation of constants: e^((t-s)A) inside, integrate in s, and do not forget e^(tA) x(0)

In $x(t) = e^{tA}x(0) + \int_0^te^{(t-s)A}g(s)\,ds$ the matrix inside the integral has $t - s$ in its exponent, $g$ is evaluated at $s$ , and $t$ is a constant for the integration. Dropping the first term (homogeneous part) is the commonest slip.

Watch out· This matrix norm is the sum of all absolute values, not the operator norm

Worksheet 24 uses $\lVert A\rVert = \sum_{i,j}\lvert a_{ij}\rvert$ . For $A = \begin{bmatrix}1 & -2\\3 & 0\end{bmatrix}$ this is $6$ . It is not the largest singular value and not the largest row or column sum. It is submultiplicative, which is the property the convergence proof needs.

Problems

checkMATH 376 worksheet 24 #1

Compute $\lVert A\rVert$ for $A = \begin{bmatrix}1 & -2\\3 & 0\end{bmatrix}$ and for $B = \begin{bmatrix}1 & 1 & 0\\-1 & 2 & 4\end{bmatrix}$ . Bound $\lVert A^3\rVert$ using submultiplicativity.

Solution

$\lVert A\rVert = 1 + 2 + 3 + 0 = 6$ and $\lVert B\rVert = 1 + 1 + 0 + 1 + 2 + 4 = 9$ . By submultiplicativity $\lVert A^3\rVert \le 6^3 = 216$ . (The true value is $\lVert A^3\rVert = 42$ , with $A^2 = \begin{bmatrix}-5 & -2\\3 & -6\end{bmatrix}$ and $A^3 = \begin{bmatrix}-11 & 10\\-15 & -6\end{bmatrix}$ ; the bound is not tight.)

checkMATH 376 worksheet 24 #6(a), worksheet 25 #1

Give $e^{tA}$ and the general solution of $x' = Ax$ for $A = \operatorname{diag}(2, -1)$ .

Solution

$e^{tA} = \operatorname{diag}(e^{2t}, e^{-t})$ , and with $x(0) = (c_1, c_2)$ : $x(t) = (e^{2t}c_1, e^{-t}c_2)$ . One coordinate grows, the other decays: a saddle.

checkMATH 376 practice final 7(b)

True or false: for every square matrix $A$ , $e^A$ is invertible. Give the reason.

Solution

True. $A$ and $-A$ commute, so $e^Ae^{-A} = e^{A + (-A)} = e^0 = I$ . Hence $(e^A)^{-1} = e^{-A}$ . (Also $\det e^A = e^{\operatorname{tr}A} \ne 0$ .)

check

Classify the origin for $x' = Ax$ : (a) $A = \begin{bmatrix}-1 & 4\\1 & -1\end{bmatrix}$ ; (b) $A = \begin{bmatrix}0 & 3\\-3 & 0\end{bmatrix}$ ; (c) $A = \begin{bmatrix}-2 & 0\\0 & -2\end{bmatrix}$ .

Solution

(a) $\tau = -2$ , $\Delta = 1 - 4 = -3 < 0$ : a saddle (eigenvalues $-1 \pm 2$ , i.e. $1$ and $-3$ ). (b) $\tau = 0$ , $\Delta = 9$ : a centre (eigenvalues $\pm3i$ ); orbits are circles traversed clockwise. (c) $\tau = -4$ , $\Delta = 4$ , $\tau^2 = 4\Delta$ : a stable degenerate node (a star): $A = -2I$ , every orbit is a straight line into $O$ , $e^{tA} = e^{-2t}I$ .

useMATH 376 worksheet 25 #4, #5

Solve $x' = \begin{bmatrix}3 & 1\\1 & 3\end{bmatrix}x$ , $x(0) = (1, 0)$ by the eigenvalue method, without computing $C^{-1}$ .

Solution

Eigenpairs: $4, (1, 1)$ and $2, (1, -1)$ . Write $(1, 0) = \frac12(1, 1) + \frac12(1, -1)$ . Then $x(t) = \frac12e^{4t}(1, 1) + \frac12e^{2t}(1, -1) = \frac12\big(e^{4t} + e^{2t},\ e^{4t} - e^{2t}\big)$ , the same as before. The starting vector’s coordinates in the eigenbasis are all that is needed.

useMATH 376 worksheet 25 #3

Let $A = CDC^{-1}$ with $D$ diagonal and let $x' = Ax$ , $x(0) = x_0$ . Show that $y = C^{-1}x$ satisfies $y' = Dy$ , and use this to find $x$ .

Solution

$C^{-1}$ is a constant matrix, so it passes through the derivative: $y' = C^{-1}x' = C^{-1}Ax = C^{-1}CDC^{-1}x = DC^{-1}x = Dy$ . In the eigenvector coordinates $y$ the system falls apart into $y_j' = \lambda_jy_j$ , with $y(0) = C^{-1}x_0$ . So $y(t) = e^{tD}C^{-1}x_0$ and $x = Cy = Ce^{tD}C^{-1}x_0$ , the formula of the diagonalizable theorem. For $A = \begin{bmatrix}3 & 1\\1 & 3\end{bmatrix}$ this is the computation of the stepper above.

usecourse notes on real and imaginary parts

Find two real solutions of $x' = \begin{bmatrix}0 & -1\\1 & 0\end{bmatrix}x$ by taking real and imaginary parts of $e^{\lambda t}v$ , and check they agree with $e^{tJ}$ .

Solution

The eigenvalue $\lambda = i$ has eigenvector $v = (1, -i)$ (first row: $-iv_1 - v_2 = 0$ ). So $u = (1, 0)$ , $w = (0, -1)$ , $a = 0$ , $b = 1$ . Then $x_1 = \cos t\,u - \sin t\,w = (\cos t, \sin t)$ and $x_2 = \sin t\,u + \cos t\,w = (\sin t, -\cos t)$ . The first is the first column of $e^{tJ}$ , the rotation matrix. The second is minus its second column, $-(-\sin t, \cos t)$ . Both are combinations of the columns of $e^{tJ}$ , as they must be.

useMATH 376 worksheet 26 #4; new numbers

Solve $x' = \begin{bmatrix}0 & -2\\2 & 0\end{bmatrix}x$ with $x(0) = (1, 1)$ , and describe the orbit.

Solution

$A = 0I + 2J$ , so $e^{tA} = \begin{bmatrix}\cos2t & -\sin2t\\\sin2t & \cos2t\end{bmatrix}$ and $x(t) = (\cos2t - \sin2t,\ \sin2t + \cos2t)$ . The length of $x$ is constant, $\sqrt2$ : the point goes counter-clockwise round the circle of radius $\sqrt2$ with angular speed $2$ , one lap every $\pi$ seconds.

useMATH 376 final 5(b) variant

Solve $x' = \begin{bmatrix}1 & -5\\1 & -3\end{bmatrix}x$ with $x(0) = (1, 0)$ , by the shortcut of this chapter.

Solution

Trace $-2$ , determinant $-3 + 5 = 2$ , so $\lambda^2 + 2\lambda + 2 = 0$ and $\lambda = -1 \pm i$ : $a = -1$ , $b = 1$ . $A + I = \begin{bmatrix}2 & -5\\1 & -2\end{bmatrix}$ and $(A + I)^2 = \begin{bmatrix}4 - 5 & -10 + 10\\2 - 2 & -5 + 4\end{bmatrix} = -I$ . So $e^{tA} = e^{-t}\big(\cos t\,I + \sin t\,(A + I)\big)$ and $x(0) = (1, 0)$ gives $x(t) = e^{-t}\big(\cos t + 2\sin t,\ \sin t\big)$ . A stable spiral.

useMATH 376 worksheet 26 #1, #2 style

Solve $x' = \begin{bmatrix}-1 & 0\\0 & -2\end{bmatrix}x + \begin{bmatrix}1\\e^{-2t}\end{bmatrix}$ with $x(0) = (0, 0)$ .

Solution

$e^{(t-s)A} = \operatorname{diag}(e^{-(t-s)}, e^{-2(t-s)})$ . $x_1 = \int_0^te^{-(t-s)}\cdot1\,ds = e^{-t}(e^t - 1) = 1 - e^{-t}$ . $x_2 = \int_0^te^{-2(t-s)}e^{-2s}\,ds = e^{-2t}\int_0^t1\,ds = te^{-2t}$ . (The forcing in the second coordinate is at the natural rate $e^{-2t}$ : resonance, hence the factor $t$ .) Check: $x_2' = e^{-2t} - 2te^{-2t} = -2x_2 + e^{-2t}$ .

proveMATH 376 worksheet 24 #2

Prove: if $\sum_k\lVert M_k\rVert$ converges, then $\sum_kM_k$ converges in the space of $n \times n$ matrices.

Solution

Plan. Show the partial sums are Cauchy, then use completeness.

$\lVert S_m - S_n\rVert \le \sum_{k=n+1}^{m}\lVert M_k\rVert$ , which is as small as we like for large $n$ because the tails of the convergent number series go to $0$ . So $(S_N)$ is Cauchy in $\mathbb R^{n^2}$ , which is complete (4.2). Hence $S_N$ converges. $\blacksquare$

proveMATH 376 worksheet 24 #5, homework 10 #4

Prove that $e^{(s+t)A} = e^{sA}e^{tA}$ using only the series (not the uniqueness theorem).

Solution

Plan. Multiply the two absolutely convergent series and group by total degree; the binomial theorem applies because $sA$ and $tA$ commute.

$e^{sA}e^{tA} = \sum_{p,q\ge0}\frac{s^pt^qA^{p+q}}{p!\,q!}$ (a Cauchy product of two absolutely convergent series, justified as in 1.6 applied entrywise). Group by $m = p + q$ : $\sum_m\frac{A^m}{m!}\sum_{p=0}^{m}\binom mp s^pt^{m-p} = \sum_m\frac{(s+t)^mA^m}{m!} = e^{(s+t)A}$ . The binomial theorem $(s + t)^m = \sum\binom mp s^pt^{m-p}$ is for numbers, so no commutativity of matrices is needed here: all the matrices are powers of the SAME $A$ . $\blacksquare$

proveMATH 376 worksheet 25 #2, #6

Prove that if $A = CDC^{-1}$ then $A^k = CD^kC^{-1}$ for all $k \ge 0$ , deduce $e^{tA} = Ce^{tD}C^{-1}$ , and prove that if $Av = \lambda v$ then $e^{tA}v = e^{\lambda t}v$ .

Solution

Plan. Induction for the power; then the series; then apply the series to $v$ .

$k = 0$ : both sides are $I$ . If $A^k = CD^kC^{-1}$ then $A^{k+1} = A^kA = CD^kC^{-1}CDC^{-1} = CD^{k+1}C^{-1}$ . So $e^{tA} = \sum\frac{t^kCD^kC^{-1}}{k!} = C\big(\sum\frac{t^kD^k}{k!}\big)C^{-1} = Ce^{tD}C^{-1}$ . If $Av = \lambda v$ then $A^kv = \lambda^kv$ by induction, so $e^{tA}v = \sum\frac{t^kA^kv}{k!} = \sum\frac{(\lambda t)^k}{k!}v = e^{\lambda t}v$ . $\blacksquare$

proveMATH 376 practice final 7(f)

Let $\lVert A\rVert = \sum_{i,j}\lvert a_{ij}\rvert < 1$ . Show that $I - A$ is invertible and $(I - A)^{-1} = \sum_{k\ge0}A^k$ .

Solution

Plan. Absolute convergence by submultiplicativity, then telescope.

$\lVert A^k\rVert \le \lVert A\rVert^k$ , a geometric series with ratio below $1$ , so $S = \sum A^k$ converges absolutely. $(I - A)\sum_{k\le N}A^k = I - A^{N+1} \to I$ since $\lVert A^{N+1}\rVert \le \lVert A\rVert^{N+1} \to 0$ . So $(I - A)S = I$ , and likewise $S(I - A) = I$ . $\blacksquare$ (Submultiplicativity: $\big\lvert\sum_ka_{ik}b_{kj}\big\rvert \le \sum_k\lvert a_{ik}\rvert\lvert b_{kj}\rvert$ , then sum over $i, j$ and enlarge the sum to independent indices.)

proveMATH 376 homework 10 #5

Show that there is a constant $C$ with $\lvert\log(k!) - k\log k + k\rvert \le C\log k$ for all $k \ge 2$ . Deduce that $(k!)^{1/k} \ge k/e$ , and hence that $(k!)^{1/k} \to \infty$ . (This is the fact the lecture uses to show that the series for $e^{tA}$ has infinite radius of convergence by the root test.)

Solution

Plan. Compare the sum $\log k! = \sum_{i \le k}\log i$ with the integral of $\log$ , using that $\log$ is increasing.

On $[i-1, i]$ we have $\log t \le \log i$ , so $\log i \ge \int_{i-1}^{i}\log t\,dt$ . Add over $i = 2, \dots, k$ : $\log k! \ge \int_1^k\log t\,dt = k\log k - k + 1$ . On $[i, i+1]$ we have $\log t \ge \log i$ , so $\log i \le \int_i^{i+1}\log t\,dt$ . Add over $i = 1, \dots, k$ : $\log k! \le \int_1^{k+1}\log t\,dt = (k+1)\log(k+1) - k$ . So

1 \le \log k! - k\log k + k \le (k+1)\log(k+1) - k\log k = k\log\Big(1 + \frac1k\Big) + \log(k+1).

Now $k\log(1 + \frac1k) \le 1$ (because $\log(1 + x) \le x$ ), and $\log(k+1) \le \log k^2 = 2\log k$ for $k \ge 2$ . Also $1 \le \frac{\log k}{\log 2}$ for $k \ge 2$ . So the middle quantity lies between $1$ and $1 + 2\log k \le (2 + \frac1{\log 2})\log k \le 4\log k$ , and $C = 4$ works. For the second claim, the lower bound $\log k! \ge k\log k - k$ gives $k! \ge (k/e)^k$ and so $(k!)^{1/k} \ge k/e \to \infty$ . $\blacksquare$

stretch

Prove that for a real $2 \times 2$ matrix with complex eigenvalues $a \pm ib$ ( $b \ne 0$ ), every solution of $x' = Ax$ has the form $x(t) = e^{at}\big(\cos bt\,u + \sin bt\,w\big)$ for fixed real vectors $u, w$ , and find $u$ and $w$ in terms of $x(0)$ .

Solution

By the $2\times2$ theorem $e^{tA}x(0) = e^{at}\big(\cos bt\,x(0) + \sin bt\,\frac1b(A - aI)x(0)\big)$ . So $u = x(0)$ and $w = \frac1b(A - aI)x(0)$ . Both are real vectors. For the final’s 5(b) matrix with $x(0) = (c_1, c_2)$ : $u = (c_1, c_2)$ and $w = \frac1{\sqrt3}(-c_1 + 2c_2,\ -2c_1 + c_2)$ , matching the formulas found there. $\blacksquare$

stretch

Show that the origin is a centre if and only if $\operatorname{tr}A = 0$ and $\det A > 0$ , and that in that case the function $V(x) = x^{\mathsf T}Mx$ is constant along solutions for some positive definite matrix $M$ . (Hint: for $A = \begin{bmatrix}a & b\\c & -a\end{bmatrix}$ try $M = \begin{bmatrix}-c & a\\a & b\end{bmatrix}$ when $b > 0 > c$ .)

Solution

Trace $0$ and determinant $\Delta > 0$ means eigenvalues $\pm i\sqrt\Delta$ , pure imaginary, so by the classification the orbits are closed. Conversely, the classification says a centre needs $\tau = 0 < \Delta$ . For $A = \begin{bmatrix}a & b\\c & -a\end{bmatrix}$ with $\Delta = -a^2 - bc > 0$ (so $bc < -a^2 \le 0$ , $b$ and $c$ of opposite signs) take $b > 0 > c$ (if $b < 0 < c$ , use $-M$ with the same formula for $M$ ; the computation below is unchanged and $-M$ is positive definite) and $M = \begin{bmatrix}-c & a\\a & b\end{bmatrix}$ , which is positive definite ( $-c > 0$ and $\det M = -bc - a^2 = \Delta > 0$ , 3.2). Then $\frac{d}{dt}x^{\mathsf T}Mx = x'^{\mathsf T}Mx + x^{\mathsf T}Mx' = x^{\mathsf T}(A^{\mathsf T}M + MA)x$ . Multiplying out the four products, $A^{\mathsf T}M + MA$ is the zero matrix (for instance, in position $(1, 1)$ the product $A^{\mathsf T}M$ gives $a(-c) + ca = 0$ and $MA$ gives $(-c)a + ac = 0$ ; the other three positions cancel in the same way), so $V$ is constant along solutions: the orbits are the ellipses $V =$ const. $\blacksquare$

Summary

A system $x' = Ax$ with constant $A$ is solved by $x(t) = e^{(t-a)A}x(a)$ , where the matrix exponential $e^{tA} = \sum t^kA^k/k!$ converges for every matrix, because $\lVert A^k\rVert \le \lVert A\rVert^k$ (submultiplicativity of the sum-of-entries norm) and $k!$ beats any exponential.
Properties: $\frac{d}{dt}e^{tA} = Ae^{tA}$ (term-by-term differentiation); $e^{(s+t)A} = e^{sA}e^{tA}$ and $e^{tA}$ is invertible with inverse $e^{-tA}$ (uniqueness of solutions, 7.1). $e^{A+B} = e^Ae^B$ only if $AB = BA$ , and $e^{tA}$ is never the entrywise exponential.
Computing $e^{tA}$ : diagonal, entrywise; diagonalizable, $Ce^{tD}C^{-1}$ (or the eigenvalue method $x = \sum c_je^{\lambda_jt}v_j$ ); scalar plus nilpotent, $e^{\lambda t}(I + tN + \cdots)$ (a Jordan block); real $2\times2$ with eigenvalues $a \pm ib$ , $e^{at}\big(\cos bt\,I + \frac{\sin bt}{b}(A - aI)\big)$ because $(A - aI)^2 = -b^2I$ ; and in general Putzer’s method from the eigenvalues alone, proved with Cayley–Hamilton.
Inhomogeneous: $x(t) = e^{(t-a)A}b + \int_a^te^{(t-s)A}g(s)\,ds$ (variation of constants), the system form of 7.3.
A power series ansatz recovers $e^{rt}$ , $e^{x^2/2}$ and $\sin t$ ; the Neumann series gives $(I - A)^{-1} = \sum A^k$ when $\lVert A\rVert < 1$ .
Phase portraits in the plane: trace $\tau$ and determinant $\Delta$ decide saddle ( $\Delta < 0$ ), node, spiral, centre ( $\tau = 0 < \Delta$ ); everything decays exactly when $\tau < 0 < \Delta$ . Orbits never cross (uniqueness).
Your 376 trap: on the final’s 5(b) you wrote only the matrix. Write eigenvalues and the form of the real solution first, then the shortcut $(A - aI)^2 = -b^2I$ finishes it in four lines.

Watch and read

The same ideas explained by someone else. Use these when an explanation here does not land.

7.4 Systems of equations and the matrix exponential

The picture

The idea, in words

Definitions and theorems

Systems of differential equations

System of differential equations

Eigenvectors give solutions

Norms and convergence of matrix series

Matrix norm

The matrix norm is submultiplicative

Absolutely convergent matrix series converge

The matrix exponential

Matrix exponential

The matrix exponential always converges

The derivative of e^(tA)

The law of exponents for one matrix

Solution of the linear system

e^(A+B) equals e^A e^B when A and B commute

e^A is always invertible

Computing e^(tA)

The exponential of a diagonal matrix

Exponential of a diagonalizable matrix

Exponential of a rotation-scaling block

Exponential of a 2 by 2 matrix

Real and imaginary parts of a complex solution

Nilpotent matrix

Exponential of a scalar plus a nilpotent matrix

Putzer's method

Inhomogeneous systems

Variation of constants for systems

Power series and the Neumann series

The Neumann series

Phase portraits: a first look

Classification of planar linear systems

Where it is used

Traps

Problems

Summary

Watch and read

Where this sits

Comes from

Leads to

Same idea elsewhere