Eigenvalues and Eigenvectors

If A is an n by n real-valued matrix we say that λ is an eigenvalue of A with associated eigenvector x if

A x = λ x

Typically, a matrix A will have several different eigenvalue, eigenvector pairs. The standard method used to find these pairs is to note that

A x = λ x

if and only if

(A - λ I) x = 0

which will happen if and only if the matrix A - λ I is singular. One way to determine whether or not A - λ I is singular is to compute its determinant and check whether or not that determinant is 0. det(A - λ I) is an nth degree polynomial in λ (called the characteristic polynomial of A), so the problem of finding eigenvalues boils down to the problem of finding roots of that polynomial.

There are a number of technical problems associated with finding eigenvalue, eigenvector pairs.

  1. Finding all the roots of a polynomial of high degree may be technically challenging.
  2. Not all roots of the characteristic polynomial may be real. Even though all of the entries of A are real, A may still have some complex eigenvalues. Complex eigenvalues will also have complex eigenvectors associated with them.
  3. Some roots may be repeated. In the case of repeated roots with multiplicity k we may be able to find k linearly independent eigenvectors for that eigenvalue, or we may be able to find fewer than k associated eigenvectors.

Basis of Eigenvectors

Assuming for the moment that we can find a set of n distinct eigenvectors for some matrix A and also assuming that those eigenvectors form an orthonormal set, we can use those eigenvectors as a basis for the vector space n. It turns out that such a basis is especially well suited to help us solve the matrix equation

A x = b

The solution method consists of expressing both x and b as linear combinations of eigenvectors.

A (c1 u1 + c2 u2 + ⋯ + cn un) = d1 u1 + d2 u2 + ⋯ + dn un

Since multiplication by A is a linear operation and the vectors uk are all eigenvectors of A we have

c1 λ1 u1 + c2 λ2 u2 + ⋯ + cn λn un = d1 u1 + d2 u2 + ⋯ + dn un

Further, since we are assuming that the eigenvectors form a basis and are hence linearly independent, this equation has a solution if and only if

ck λk = dk

for all k. Since both λk and dk are known, if none of the eigenvalues λk are 0 we can solve these equations for all of the ck and then construct

x = c1 u1 + c2 u2 + ⋯ + cn un

This method, which uses a basis of eigenvectors and their associated eigenvalues to solve a linear system is called the spectral method. (The list of eigenvalues of an operator is sometimes referred to as the spectrum of that operator, hence spectral method.)

The significance of this method is that it applies more broadly to problems involving linear operators. To solve

f(x) = b

for a linear operator f we try to find a basis of eigenvectors and associated eigenvalues:

f(uk) = λk uk

To solve f(x) = b we then proceed as above:

f(c1 u1 + c2 u2 + ⋯ + cn un) = d1 u1 + d2 u2 + ⋯ + dn un

c1 λ1 u1 + c2 λ2 u2 + ⋯ + cn λn un = d1 u1 + d2 u2 + ⋯ + dn un

This can be solved as above by solving the equations

ck λk = dk

for the unknowns ck.

An Important Special Case

There is one very important special class of matrices for which the program outlined above works very nicely. These are the n by n symmetric, real-valued matrices. A matrix A is symmetric if it is equal to its own transpose. The key theorem that tells us that real-valued symmetric matrices are nice is the following.

Theorem If A is a real-valued, symmetric, n by n matrix, A has a complete set of associated real-valued eigenvectors. Further, eigenvectors corresponding to distinct eigenvalues are orthogonal to each other. Thus, it is possible to construct an orthonormal basis for n consisting of eigenvectors of A.