Inverse matrix, the [A | I] method and elementary matrices

With numbers you solve the equation $3x = 6$ by multiplying both sides by $3^{-1}$. For matrices the same idea is called the inverse matrix: if $A\mathbf{x} = \mathbf{b}$ then $\mathbf{x} = A^{-1}\mathbf{b}$. But not every matrix has an inverse, and it matters from which side you multiply. On the midterm you will almost certainly get a $2 \times 2$ formula question, a $3 \times 3$ $[A \mid I]$ question and an $AX = B$ question from this topic.

Golden rules:

1. The inverse is defined only for square matrices.
2. For $2 \times 2$ use the formula, for $3 \times 3$ the $[A \mid I]$ method.
3. If $AX = B$ then $X = A^{-1}B$; if $XA = B$ then $X = BA^{-1}$. Mixing up the side is the most common mistake.
4. Always check: a row · a column must give $1$ or $0$.

Definition

For an $n \times n$ matrix $A$, if there is a $B$ with

$AB = BA = I_n$

then $A$ is invertible (nonsingular) and $B$ is the inverse $A^{-1}$ of $A$. If there is no such $B$, $A$ is called singular.

The inverse is unique. If $B$ and $C$ are both inverses of $A$, then $B = BI = B(AC) = (BA)C = IC = C$.

For square matrices one side is enough: if $AB = I$ then automatically $BA = I$. That is why the problem book has the question “AB = I ⇒ BA = I”.

The 2×2 formula

$A = \begin{bmatrix} a & b \\ c & d \end{bmatrix}, \quad ad - bc \neq 0 \quad\Rightarrow\quad A^{-1} = \frac{1}{ad - bc}\begin{bmatrix} d & -b \\ -c & a \end{bmatrix}.$

Recipe to memorize: swap the diagonal, flip the signs of the other two, divide by $ad - bc$. If $ad - bc = 0$ there is no inverse. This number is the $2 \times 2$ determinant (next week).

Example: $A = \begin{bmatrix} 5 & 2 \\ 7 & 3 \end{bmatrix}$, $ad - bc = 15 - 14 = 1$:

$A^{-1} = \begin{bmatrix} 3 & -2 \\ -7 & 5 \end{bmatrix}. \qquad \text{Check: } \begin{bmatrix} 5 & 2 \\ 7 & 3 \end{bmatrix}\begin{bmatrix} 3 & -2 \\ -7 & 5 \end{bmatrix} = \begin{bmatrix} 15 - 14 & -10 + 10 \\ 21 - 21 & -14 + 15 \end{bmatrix} = I.$

The [A | I] method

Why does it work? All the row operations that take $A$ to $I$ can be thought of as a product: $E_k \cdots E_2E_1A = I$. This product is exactly $A^{-1}$. If you apply the same operations to $I$ on the side, you get $E_k \cdots E_1 I = A^{-1}$.

Recipe:

1. Write the $n \times 2n$ matrix: $[A \mid I]$.
2. Reduce the left side to $I$ with Gauss-Jordan. Apply every operation to all $2n$ columns.
3. If the left side becomes $I$, the right side is $A^{-1}$. If a zero row appears on the left, $A$ is singular; stop.

Example. $A = \begin{bmatrix} 1 & 2 & 0 \\ 2 & 5 & 1 \\ 1 & 2 & 1 \end{bmatrix}$.

$\left[\begin{array}{ccc|ccc} 1 & 2 & 0 & 1 & 0 & 0 \\ 2 & 5 & 1 & 0 & 1 & 0 \\ 1 & 2 & 1 & 0 & 0 & 1 \end{array}\right] \xrightarrow[R_3 \leftarrow R_3 - R_1]{R_2 \leftarrow R_2 - 2R_1} \left[\begin{array}{ccc|ccc} 1 & 2 & 0 & 1 & 0 & 0 \\ 0 & 1 & 1 & -2 & 1 & 0 \\ 0 & 0 & 1 & -1 & 0 & 1 \end{array}\right]$

The left side is upper triangular and the pivots are 1. Now clear above the pivots from right to left:

$\xrightarrow{R_2 \leftarrow R_2 - R_3} \left[\begin{array}{ccc|ccc} 1 & 2 & 0 & 1 & 0 & 0 \\ 0 & 1 & 0 & -1 & 1 & -1 \\ 0 & 0 & 1 & -1 & 0 & 1 \end{array}\right] \xrightarrow{R_1 \leftarrow R_1 - 2R_2} \left[\begin{array}{ccc|ccc} 1 & 0 & 0 & 3 & -2 & 2 \\ 0 & 1 & 0 & -1 & 1 & -1 \\ 0 & 0 & 1 & -1 & 0 & 1 \end{array}\right]$

$A^{-1} = \begin{bmatrix} 3 & -2 & 2 \\ -1 & 1 & -1 \\ -1 & 0 & 1 \end{bmatrix}.$

check: row 1 of A (1, 2, 0) · column 1 of A⁻¹ (3, -1, -1) = 3 - 2 + 0 = 1   ✓
       row 2 of A (2, 5, 1) · column 1 of A⁻¹ (3, -1, -1) = 6 - 5 - 1 = 0   ✓

A singular example. $S = \begin{bmatrix} 1 & 0 & 2 \\ 2 & 1 & 3 \\ 0 & 1 & -1 \end{bmatrix}$:

$\left[\begin{array}{ccc|ccc} 1 & 0 & 2 & 1 & 0 & 0 \\ 2 & 1 & 3 & 0 & 1 & 0 \\ 0 & 1 & -1 & 0 & 0 & 1 \end{array}\right] \xrightarrow{R_2 \leftarrow R_2 - 2R_1} \left[\begin{array}{ccc|ccc} 1 & 0 & 2 & 1 & 0 & 0 \\ 0 & 1 & -1 & -2 & 1 & 0 \\ 0 & 1 & -1 & 0 & 0 & 1 \end{array}\right] \xrightarrow{R_3 \leftarrow R_3 - R_2} \left[\begin{array}{ccc|ccc} 1 & 0 & 2 & 1 & 0 & 0 \\ 0 & 1 & -1 & -2 & 1 & 0 \\ 0 & 0 & 0 & 2 & -1 & 1 \end{array}\right]$

A zero row has appeared on the left side. Whatever you do, there will be no pivot in row 3: $S$ is singular. The fact that the right side is not zero changes nothing; the decision is made by looking at the left side.

Solving a system with the inverse

If $A$ is invertible, $A\mathbf{x} = \mathbf{b}$ has a unique solution: $\mathbf{x} = A^{-1}\mathbf{b}$. For the $A$ above and $\mathbf{b} = (1, 3, 2)^T$:

$\mathbf{x} = \begin{bmatrix} 3 & -2 & 2 \\ -1 & 1 & -1 \\ -1 & 0 & 1 \end{bmatrix}\begin{bmatrix} 1 \\ 3 \\ 2 \end{bmatrix} = \begin{bmatrix} 3 - 6 + 4 \\ -1 + 3 - 2 \\ -1 + 0 + 2 \end{bmatrix} = \begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix}.$

If many different $\mathbf{b}$ are to be solved with the same $A$, finding $A^{-1}$ once speeds things up. For a single system, Gaussian elimination is usually shorter.

Matrix equations. Let $A$ be invertible:

PITFALL · there is no “division”. There is no such thing as $X = \frac{B}{A}$. $A^{-1}B$ and $BA^{-1}$ are generally different (Q6 and Q7 give two different answers with the same matrices).

Properties of the inverse

$A$, $B$ invertible $n \times n$, $c \neq 0$:

Check of $(AB)^{-1} = B^{-1}A^{-1}$: $(AB)(B^{-1}A^{-1}) = A(BB^{-1})A^{-1} = AA^{-1} = I$. If the order were not reversed, $B$ and $B^{-1}$ would not meet in the middle.

The cancellation rule now holds too: if $A$ is invertible, $AB = AC \Rightarrow B = C$ (multiply from the left by $A^{-1}$).

Elementary matrices

An elementary matrix $E$ is a matrix obtained by applying one row operation to the identity matrix. Its magic property: the product $EA$ is the same as applying that row operation to $A$.

Every elementary matrix is invertible and its inverse is elementary too: the operation that undoes the operation.

Writing a matrix as a product of elementary matrices. $A = \begin{bmatrix} 1 & 2 \\ 3 & 7 \end{bmatrix}$:

A  --R2 ← R2 - 3R1-->  [1 2; 0 1]  --R1 ← R1 - 2R2-->  I
E1 = [1 0; -3 1],   E2 = [1 -2; 0 1],   E2 E1 A = I
A = E1⁻¹ E2⁻¹ = [1 0; 3 1] [1 2; 0 1]

Check: $\begin{bmatrix} 1 & 0 \\ 3 & 1 \end{bmatrix}\begin{bmatrix} 1 & 2 \\ 0 & 1 \end{bmatrix} = \begin{bmatrix} 1 & 2 \\ 3 & 7 \end{bmatrix}$. The order of the inverses reversed, because $A = (E_2E_1)^{-1} = E_1^{-1}E_2^{-1}$.

The invertible matrix theorem (first version)

If $A$ is $n \times n$, the following are equivalent (if one is true, all are true):

1. $A$ is invertible.
2. $A\mathbf{x} = \mathbf{0}$ has only the trivial solution.
3. The RREF of $A$ is $I_n$.
4. $A$ is a product of elementary matrices.
5. $A\mathbf{x} = \mathbf{b}$ is consistent with a unique solution for every $\mathbf{b}$.

The list will grow through the semester: $\det A \neq 0$ (week 4), columns independent (week 7), rank $= n$ (week 9), 0 is not an eigenvalue (week 13).

Common mistakes