Skip to main content
Logo image

Understanding Linear Algebra

David Austin

Section 3.1 Invertibility

Up to this point, we have used the Gaussian elimination algorithm to find solutions to linear systems. We now investigate another way to find solutions to the equation Ax=b when the matrix A has the same number of rows and columns. To get started, let’s look at some familiar examples.
🔗

Preview Activity 3.1.1.

  1. Explain how you would solve the equation 3x=5 using multiplication rather than division.
  2. Find the 2×2 matrix A that rotates vectors counterclockwise by 90∘.
  3. Find the 2×2 matrix B that rotates vectors clockwise by 90∘.
  4. What do you expect the product AB to be? Explain the reasoning behind your expectation and then compute AB to verify it.
  5. Solve the equation Ax=[3−2] using Gaussian elimination.
  6. Explain why your solution may also be found by computing x=B[3−2].
🔗
🔗

Subsection 3.1.1 Invertible matrices

The preview activity began with a familiar type of equation, 3x=5, and asked for a strategy to solve it. One possible response is to divide both sides by 3. Instead, let’s rephrase this as multiplying by 3−1=13, the multiplicative inverse of 3.
🔗
Now that we are interested in solving equations of the form Ax=b, we might try to find a similar approach. Is there a matrix A−1 that plays the role of the multiplicative inverse of A? Of course, the real number 0 does not have a multiplicative inverse so we probably shouldn’t expect every matrix to have a multiplicative inverse. We will see, however, that many do.
🔗

Definition 3.1.1.

An n×n matrix A is called invertible if there is a matrix B such that AB=In, where In is the n×n identity matrix. The matrix B is called the inverse of A and denoted A−1.
🔗
🔗
Notice that we only define invertibility for matrices that have the same number of rows and columns in which case we say that the matrix is square.
🔗

Example 3.1.2.

Suppose that A is the matrix that rotates two-dimensional vectors counterclockwise by 90∘ and that B rotates vectors by −90∘. We have
🔗
A=[0−110],   B=[01−10].
We can check that
🔗
AB=[0−110][01−10]=[1001]=I
which shows that A is invertible and that A−1=B.
🔗
🔗
Notice that if we multiply the matrices in the opposite order, we find that BA=I, which says that B is also invertible and that B−1=A. In other words, A and B are inverses of each other.
🔗
🔗

Activity 3.1.2.

This activity demonstrates a procedure for finding the inverse of a matrix A.
🔗
  1. Suppose that A=[3−21−1]. To find an inverse B, we write its columns as B=[b1b2] and require that
    AB=I[Ab1Ab2]=[1001].
    In other words, we can find the columns of B by solving the equations
    Ab1=[10],   Ab2=[01].
    Solve these equations to find b1 and b2. Then write the matrix B and verify that AB=I. This is enough for us to conclude that B is the inverse of A.
  2. Find the product BA and explain why we now know that B is invertible and B−1=A.
  3. What happens when you try to find the inverse of C=[−214−2]?
  4. We now develop a condition that must be satisfied by an invertible matrix. Suppose that A is an invertible n×n matrix with inverse B and suppose that b is any n-dimensional vector. Since AB=I, we have
    A(Bb)=(AB)b=Ib=b.
    This says that the equation Ax=b is consistent and that x=Bb is a solution.
    Since we know that Ax=b is consistent for any vector b, what does this say about the span of the columns of A?
  5. Since A is a square matrix, what does this say about the pivot positions of A? What is the reduced row echelon form of A?
  6. In this activity, we have studied the matrices
    A=[3−21−1],   C=[−214−2].
    Find the reduced row echelon form of each and explain how those forms enable us to conclude that one matrix is invertible and the other is not.
🔗
🔗

Example 3.1.3.

We can reformulate this procedure for finding the inverse of a matrix. For the sake of convenience, suppose that A is a 2×2 invertible matrix with inverse B=[b1b2]. Rather than solving the equations
🔗
Ab1=[10],   Ab2=[01]
separately, we can solve them at the same time by augmenting A by both vectors [10] and [01] and finding the reduced row echelon form.
🔗
🔗
For example, if A=[1211], we form
🔗
[12101101]∼[10−12011−1].
This shows that the matrix B=[−1211] is the inverse of A.
🔗
🔗
In other words, beginning with A, we augment by the identify and find the reduced row echelon form to determine A−1:
🔗
[AI]∼[IA−1].
🔗
🔗
In fact, this reformulation will always work. Suppose that A is an invertible n×n matrix with inverse B. Suppose furthermore that b is any n-dimensional vector and consider the equation Ax=b. We know that x=Bb is a solution because A(Bb)=(AB)b=Ib=b.
🔗
Notice that this is similar to saying that the solution to 3x=5 is x=13⋅5, as we saw in the preview activity.
🔗
Now since Ax=b is consistent for every vector b, the columns of A must span Rn so there is a pivot position in every row. Since A is also square, this means that the reduced row echelon form of A is the identity matrix.
🔗
You may have noticed that Proposition 3.1.4 says that the solution to the equation Ax=b is x=A−1b. Indeed, we know that this equation has a unique solution because A has a pivot position in every column.
🔗
It is important to remember that the product of two matrices depends on the order in which they are multiplied. That is, if C and D are matrices, then it sometimes happens that CD≠DC. However, something fortunate happens when we consider invertibility. It turns out that if A is an n×n matrix and that AB=I, then it is also true that BA=I. We have verified this in a few examples so far, and Exercise 3.1.5.12 explains why it always happens. This leads to the following proposition.
🔗
🔗

Subsection 3.1.2 Solving equations with an inverse

If A is an invertible matrix, then Proposition 3.1.4 shows us how to use A−1 to solve equations involving A. In particular, the solution to Ax=b is x=A−1b.
🔗

Activity 3.1.3.

We’ll begin by considering the square matrix
🔗
A=[102221111].
  1. Describe the solution space to the equation Ax=[343] by augmenting A and finding the reduced row echelon form.
  2. Using Proposition 3.1.5, explain why A is invertible and find its inverse.
  3. Now use the inverse to solve the equation Ax=[343] and verify that your result agrees with what you found in part a.
  4. If you have defined a matrix B in Sage, you can find it’s inverse as B.inverse() or B^-1. Use Sage to find the inverse of the matrix
    B=[1−2−1−1565−46]
    and use it to solve the equation Bx=[8336].
  5. If A and B are the two matrices defined in this activity, find their product AB and verify that it is invertible.
  6. Compute the products A−1B−1 and B−1A−1. Which one agrees with (AB)−1?
  7. Explain your finding by considering the product
    (AB)(B−1A−1)
    and using associativity to regroup the products so that the middle two terms are multiplied first.
🔗
🔗
The next proposition summarizes much of what we have found about invertible matrices.
🔗
There is a simple formula for finding the inverse of a 2×2 matrix:
🔗
[abcd]−1=1ad−bc[d−b−ca],
which can be easily checked. The condition that A be invertible is, in this case, reduced to the condition that ad−bc≠0. We will understand this condition better once we have explored determinants in Section 3.4. There is a similar formula for the inverse of a 3×3 matrix, but there is not a good reason to write it here.
🔗
🔗
🔗

Subsection 3.1.3 Triangular matrices and Gaussian elimination

With some of the ideas we’ve developed, we can recast the Gaussian elimination algorithm in terms of matrix multiplication and invertibility. This will be especially helpful later when we consider determinants and LU factorizations. Triangular matrices will play an important role.
🔗

Definition 3.1.8.

We say that a matrix A is lower triangular if all its entries above the diagonal are zero. Similarly, A is upper triangular if all the entries below the diagonal are zero.
🔗
🔗
For example, the matrix L below is a lower triangular matrix while U is an upper triangular one.
🔗
L=[∗000∗∗00∗∗∗0∗∗∗∗],U=[∗∗∗∗0∗∗∗00∗∗000∗].
🔗
We can develop a simple test to determine whether an n×n lower triangular matrix is invertible. Let’s use Gaussian elimination to find the reduced row echelon form of the lower triangular matrix
🔗
[1002−20−34−4]∼[1000−2004−4]∼[1000−2000−4]∼[100010001].
Because the entries on the diagonal are nonzero, we find a pivot position in every row, which tells us that the matrix is invertible.
🔗
🔗
If, however, there is a zero entry on the diagonal, the matrix cannot be invertible. Considering the matrix below, we see that having a zero on the diagonal leads to a row without a pivot position.
🔗
[100200−34−4]∼[10000004−4]∼[10001−1000].
🔗

Activity 3.1.4. Gaussian elimination and matrix multiplication.

This activity explores how the row operations of scaling, interchange, and replacement can be performed using matrix multiplication.
🔗
As an example, we consider the matrix
🔗
A=[12120−2−12−1]
and apply a replacement operation that multiplies the first row by −2 and adds it to the second row. Rather than performing this operation in the usual way, we construct a new matrix by applying the desired replacement operation to the identity matrix. To illustrate, we begin with the identity matrix
🔗
I=[100010001]
and form a new matrix by multiplying the first row by −2 and adding it to the second row to obtain
🔗
R=[100−210001].
  1. Show that the product RA is the result of applying the replacement operation to A.
  2. Explain why R is invertible and find its inverse R−1.
  3. Describe the relationship between R and R−1 and use the connection to replacement operations to explain why it holds.
  4. Other row operations can be performed using a similar procedure. For instance, suppose we want to scale the second row of A by 4. Find a matrix S so that SA is the same as that obtained from the scaling operation. Why is S invertible and what is S−1?
  5. Finally, suppose we want to interchange the first and third rows of A. Find a matrix P, usually called a permutation matrix that performs this operation. What is P−1?
  6. The original matrix A is seen to be row equivalent to the upper triangular matrix U by performing three replacement operations on A:
    A=[12120−2−12−1]∼[1210−4−400−4]=U.
    Find the matrices L1, L2, and L3 that perform these row replacement operations so that L3L2L1A=U.
  7. Explain why the matrix product L3L2L1 is invertible and use this fact to write A=LU. What is the matrix L that you find? Why do you think we denote it by L?
🔗
🔗
The following are examples of matrices, known as elementary matrices, that perform the row operations on a matrix having three rows.
🔗
Replacement
Multiplying the second row by 3 and adding it to the third row is performed by
🔗
L=[100010031].
We often use L to describe these matrices because they are lower triangular.
🔗
🔗
Scaling
Multiplying the third row by 2 is performed by
🔗
S=[100010002].
🔗
Interchange
Interchanging the first two rows is performed by
🔗
P=[010100001].
🔗
🔗

Example 3.1.10.

Suppose we have
🔗
A=[13−2−3−6320−2].
For the forward substitution phase of Gaussian elimination, we perform a sequence of three replacement operations. The first replacement operation multiplies the first row by 3 and adds the result to the second row. We can perform this operation by multiplying A by the lower triangular matrix L1 where
🔗
L1A=[100310001][13−2−3−6320−2]=[13−203−320−2].
🔗
The next two replacement operations are performed by the matrices
🔗
L2=[100010−201],L3=[100010021]
so that
🔗
L3L2L1A=U=[13−203−300−4].
🔗
Notice that the inverse of L1 has the simple form:
🔗
L1=[100310001],L1−1=[100−310001].
This says that if we want to undo the operation of multiplying the first row by 3 and adding to the second row, we should multiply the first row by −3 and add it to the second row. That is the effect of L1−1.
🔗
🔗
Notice that we now have L3L2L1A=U, which gives
🔗
(L3L2L1)A=U(L3L2L1)−1(L3L2L1)A=(L3L2L1)−1UA=(L3L2L1)−1U=LU
where L is the lower triangular matrix
🔗
L=(L3L2L1)−1=[100−3102−21].
This way of writing A=LU as the product of a lower and an upper triangular matrix is known as an LU factorization of A, and its usefulness will be explored in Section 5.1.
🔗
🔗
🔗
🔗

Subsection 3.1.4 Summary

In this section, we found conditions guaranteeing that a matrix has an inverse. When these conditions hold, we also found an algorithm for finding the inverse.
🔗
  • A square matrix is invertible if there is a matrix B, known as the inverse of A, such that AB=I. We usually write A−1=B.
  • The n×n matrix A is invertible if and only if it is row equivalent to In, the n×n identity matrix.
  • If a matrix A is invertible, we can use Gaussian elimination to find its inverse:
    [AI]∼[IA−1].
  • If a matrix A is invertible, then the solution to the equation Ax=b is x=A−1b.
  • The row operations of replacement, scaling, and interchange can be performed by multiplying by elementary matrices.
🔗
🔗

Exercises 3.1.5 Exercises

1.

Consider the matrix
🔗
A=[3−1140231−210−23012].
🔗
  1. Explain why A has an inverse.
  2. Find the inverse of A by augmenting by the identity I to form [AI].
  3. Use your inverse to solve the equation Ax=[32−3−1].
🔗
🔗

2.

In this exercise, we will consider 2×2 matrices as defining matrix transformations.
🔗
  1. Write the matrix A that performs a 45∘ rotation. What geometric operation undoes this rotation? Find the matrix that perform this operation and verify that it is A−1.
  2. Write the matrix A that performs a 180∘ rotation. Verify that A2=I so that A−1=A, and explain geometrically why this is the case.
  3. Find three more matrices A that satisfy A2=I.
🔗
🔗

3.

Inverses for certain types of matrices can be found in a relatively straightforward fashion.
🔗
  1. The matrix D=[2000−1000−4] is called diagonal since the only nonzero entries are on the diagonal of the matrix.
    1. Find D−1 by augmenting D by the identity and finding its reduced row echelon form.
    2. Under what conditions is a diagonal matrix invertible?
    3. Explain why the inverse of a diagonal matrix is also diagonal and explain the relationship between the diagonal entries in D and D−1.
  2. Consider the lower triangular matrix L=[100−2103−41].
    1. Find L−1 by augmenting L by the identity and finding its reduced row echelon form.
    2. Explain why the inverse of a lower triangular matrix is also lower triangular.
🔗
🔗

4.

Our definition of an invertible matrix requires that A be a square n×n matrix. Let’s examine what happens when A is not square. For instance, suppose that
🔗
A=[−1−1−2−130],B=[−2211−2−1].
  1. Verify that BA=I2. In this case, we say that B is a left inverse of A.
  2. If A has a left inverse B, we can still use it to find solutions to linear equations. If we know there is a solution to the equation Ax=b, we can multiply both sides of the equation by B to find x=Bb.
    Suppose you know there is a solution to the equation Ax=[−1−36]. Use the left inverse B to find x and verify that it is a solution.
  3. Now consider the matrix
    C=[1−10−210]
    and verify that C is also a left inverse of A. This shows that the matrix A may have more than one left inverse.
🔗
🔗

5.

If a matrix A is invertible, there is a sequence of row operations that transforms A into the identity matrix I. We have seen that every row operation can be performed by matrix multiplication. If the jth step in the Gaussian elimination process is performed by multiplying by Ej, then we have
🔗
Ep⋯E2E1A=I,
which means that
🔗
A−1=Ep⋯E2E1.
For each of the following matrices, find a sequence of row operations that transforms the matrix to the identity I. Write the matrices Ej that perform the steps and use them to find A−1.
🔗
  1. A=[020−300001].
  2. A=[100021000−3100021].
  3. A=[111011002].
🔗
🔗

6.

Suppose that A is an n×n matrix.
🔗
  1. Suppose that A2=AA is invertible with inverse B. This means that A2B=AAB=I. Explain why A must be invertible with inverse AB.
  2. Suppose that A100 is invertible with inverse B. Explain why A is invertible. What is A−1 in terms of A and B?
🔗
🔗

7.

Determine whether the following statements are true or false and explain your reasoning.
🔗
  1. If A is invertible, then the columns of A are linearly independent.
  2. If A is a square matrix whose diagonal entries are all nonzero, then A is invertible.
  3. If A is an invertible n×n matrix, then span of the columns of A is Rn.
  4. If A is invertible, then there is a nonzero solution to the homogeneous equation Ax=0.
  5. If A is an n×n matrix and the equation Ax=b has a solution for every vector b, then A is invertible.
🔗
🔗

8.

Provide a justification for your response to the following questions.
🔗
  1. Suppose that A is a square matrix with two identical columns. Can A be invertible?
  2. Suppose that A is a square matrix with two identical rows. Can A be invertible?
  3. Suppose that A is an invertible matrix and that AB=AC. Can you conclude that B=C?
  4. Suppose that A is an invertible n×n matrix. What can you say about the span of the columns of A−1?
  5. Suppose that A is an invertible matrix and that B is row equivalent to A. Can you guarantee that B is invertible?
🔗
🔗

9.

Suppose that we start with the 3×3 matrix A, perform the following sequence of row operations:
🔗
  1. Multiply row 1 by -2 and add to row 2.
  2. Multiply row 1 by 4 and add to row 3.
  3. Scale row 2 by 1/2.
  4. Multiply row 2 by -1 and add to row 3,
and arrive at the upper triangular matrix
🔗
U=[32−101300−4].
🔗
  1. Write the matrices E1, E2, E3, and E4 that perform the four row operations.
  2. Find the matrix E=E4E3E2E1.
  3. We then have E4E3E2E1A=EA=U. Now that we have the matrix E, find the original matrix A=E−1U.
🔗
🔗

10.

We say that two square matrices A and B are similar if there is an invertible matrix P such that B=PAP−1.
🔗
  1. If A and B are similar, explain why A2 and B2 are similar as well. In particular, if B=PAP−1, explain why B2=PA2P−1.
  2. If A and B are similar and A is invertible, explain why B is also invertible.
  3. If A and B are similar and both are invertible, explain why A−1 and B−1 are similar.
  4. If A is similar to B and B is similar to C, explain why A is similar to C. To begin, you may wish to assume that B=PAP−1 and C=QBQ−1.
🔗
🔗

11.

Suppose that A and B are two n×n matrices and that AB is invertible. We would like to explain why both A and B are invertible.
🔗
  1. We first explain why B is invertible.
    1. Since AB is invertible, explain why any solution to the homogeneous equation ABx=0 is x=0.
    2. Use this fact to explain why any solution to Bx=0 must be x=0.
    3. Explain why B must be invertible.
  2. Now we explain why A is invertible.
    1. Since AB is invertible, explain why the equation ABx=b is consistent for every vector b.
    2. Using the fact that ABx=A(Bx)=b is consistent for every b, explain why every equation Ax=b is consistent.
    3. Explain why A must be invertible.
🔗
🔗

12.

We defined an n×n matrix to be invertible if there is a matrix B such that AB=In. In this exercise, we will explain why it is also true that BA=I, which is the statement of Proposition 3.1.6. This means that, if B=A−1, then A=B−1.
🔗
  1. Suppose that x is an n-dimensional vector. Since AB=I, explain why ABx=x and use this to explain why the only vector for which Bx=0 is x=0.
  2. Explain why this implies that B must be invertible. We will call the inverse C so that BC=I.
  3. Beginning with AB=I, explain why B(AB)C=BIC and why this tells us that BA=I.
🔗
🔗
🔗
🔗
PrevTopNext