Eigenvalues and Eigenvectors

What are they???

Good question.  Let's get right to the point.  Let  A  be a square  n ´ n  matrix  (these are the only kind of matrices for which eigenvalues and eigenvectors will be defined).  An eigenvector of  A  is a  NONZERO  vector  x  in  Rn  (or Cn,  if we are working over complex nos),  such that for some scalar number  l, we  have

					A x  =  l x

The number l  is called an eigenvalue of  the matrix  A,  and the vector  x  is called an eigenvector belonging to (or corresponding to)  the eigenvalue  l.  Observe that  the eigenvalue  l  is allowed to be the  0  scalar,  but eigenvectors  x  are,  by definition,  never the  0  vector.  (Question:  what does it tell you about a matrix that  0  is an eigenvalue of it?)  

Mathematica  knows all about eigenvalues and eigenvectors.  Click the following cells.

a = {{4,0,1},{-2,1,0},{-2,0,1}}

MatrixForm[a]

Eigenvalues[a]

Eigenvectors[a]

Curious, isn't it?  Let's try another example.

b = {{3,2},{-1,0}}

Eigenvalues[b]

Eigenvectors[b]

Let's check some of these proposed values:

a . {0,1,0}

a. {-1,2,2}

a.{-1,1,1}

b.{-1,1}

What do you think is the number of eigenvalues of a given matrix A?   O. K.,...  try this

c = {{-2,-1},{5,2}}

Eigenvalues[c]

Eigenvectors[c]

What do you say about your answer to the previous question?

So what is the general story?  You'll find out more technical details  in class lecture,  but here are some of the basic points about eigenvalues and eigenvectors:

A  has eigenvalue  l   iff   A x  =  l x,  for some nonzero vector x
				   
			iff    A x  -  l x  =  0,   x  nonzero
				    
			iff   (A  -  l I)  x  = 0,  x nonzero
				    
			iff    x  is a nonzero element of the Null Space of  (A - l I)
				     
			iff   rank (A  -  l I)  < n  = size of  A
				     
			iff   det (A  -  l I)  = 0.

But observe that the det above is just a polynomial of degree  n  in the variable  l.  The  last equation above is called the characteristic equation of  A.  It will have exactly n roots,  if  we allow for repetitions and complex numbers.

Once we have found all eigenvalues of  A,  we can always find the appropriate Null Space and hence all eigenvectors belonging to each eigenvalue.   By the way, this Null Space has a name:  it is called the  eigenspace corresponding to the eigenvalue l.  As an example,  click the following cell:

Det[a - lambda*IdentityMatrix[3]]

Factor[Det[a-lambda*IdentityMatrix[3]]]

Det[c - lambda*IdentityMatrix[2]]

Now fill in the next cell with one of the eigenvalues of the matrix a before doing the subsequent cell:

eval = 1

NullSpace[a-eval*IdentityMatrix[3]]

So now you see that the Eigenvector command is really calculating null spaces.  Once you have calculated the null spaces you have a complete listing, in principle, of all the eigenvectors and values of the matrix.  Actually, you specify a basis for the null space of  the matrix a - lambda*IdentityMatrix[n].  This space, the eigenspace of the matrix  a corresponding to the eigenvalue lambda, has one vector in it which is not an eigenvector.  Which one is it?   A complete listing of all eigenvalues and eigenspaces of the matrix  A  is called the eigensystem of the matrix A.   In principle, you should be able to calculate eigensystems by hand.  Of course, in practice this may be difficult.  Mathematica knows about these systems and can help us out. Click the follow cells:

? Eigensystem

Eigensystem[a]

Why are they???

This  is a good question and the answer could go on and on.  We will try to indicate their importance for one special class of problems,  namely,  discrete dynamical systems.
	
Here is how such a system is described:  one starts with an initial state vector x0.  Now to pass from one state vector to the next, one uses the rule
	
					xn+1  =  A xn,
					
where  xk  means the kth state vector.  The matrix  A  is  fixed for the dynamical system.  It is called the transition matrix of the system.  You have already seen one such example,  namely,  Markov  chains.  There are others,  from fields like biology and economics.  Let's try to understand how state vectors change in the case of the matrix A from the previous section.

x0 = {1,1,1}

a.x0

a.a.x0

a.a.a.x0

a.a.a.a.x0

a.a.a.a.a.x0

What do you predict will be the next state, approximately?

What do you get if you divide the above vector by its largest component? 

Now compare these to the eigenvectors and eigenvalues of A.

Now do the whole sequence above with another random vector x0.

Next try the same thing with the matrix b.  Repeat the above type of sequence.

x0 = {1,1}

What's going on here?  Here is a start on an explanation:

Notice that x2  =  A x1 =  A A x0,

	       x3  =  A x2 = A A A x0,  etc.

See the pattern? 
				xn  =  An x0.
				
So,  to understand how a dynamical system works,  what we really need to know is how the powers of the transition matrix  A  behave.  But  in general,  this is very hard!!!

Here is a case we can handle:  what if  A  is diagonal?

d = {{1, 0, 0}, {0, 2, 0}, {0, 0, 3}}

MatrixForm[d]

MatrixForm[d.d]

MatrixForm[d.d.d]

See what's going on?   It's easy to show this

FACT:  For a (square) diagonal matrix  D  and positive integer  n  the matrix  Dn is computed from  D  by raising each diagonal entry of  D  to the  nth power.  

By the way,  one can use this fact to answer a question that we will be interested in in the next section:  When do the positive powers of  D  tend to a constant matrix?  Answer....?

Now for a more general A.  Let's take a 3 ´ 3  matrix A.  What if we could find three linearly independent eigenvectors v1, v2, and v3?  We would have

				A v1  = l1 v1
				A v2  = l2 v2
				A v3 =  l3 v3,

						   æl1   0   0  ö
Or:		A  [v1, v2, v3]  = [v1, v2, v3] . ÷  0  l2  0   ç
						   è  0   0  l3 ø

Now set   P  =  [v1, v2, v3]  and set  D = DiagonalMatrix[l1,l2, l3].   Then  P  is invertible . (Why?)  Moreover, the above equation, if multiplied on the left by P-1, gives the equation

				P-1  A  P  =  D.

Consider  D D  = (P-1 A P ) (P-1 A P ) =P-1 A A P = P-1 A2 P.
In general, the same holds for the nth powers of A and D.  Finally,  multiply on the left and right by  P  and P-1 to obtain:

KEY FACT:  If there is a matrix   P  such that  P-1 A P  =  D,  then for all positive integers n,

				An  =  P  Dn  P-1.

Call a matrix   A  diagonalizable  if there exists an invertible matrix  P  such that P-1 A P  is a diagonal matrix.   What is the story here?  Notice that all the steps we performed are reversible.  So what we have really shown  (for the case   n = 3,  but the same argument works in general) is the following important fact:

KEY FACT (Diagonalization Theorem):  Let  A  be an  n´n matrix.  Then  A  is  diagonalizable if and only if there exists a basis of eigenvectors  v1, v2, ... , vn  of A  (i.e.  v1, v2, ... , vn are linearly independent)  for  Rn.  Moreover, in this case the matrix 
P = [v1, v2, ... , vn]  is nonsingular and  P-1 A P  is  a diagonal matrix with the eigenvalues of  A along its main diagonal.

Just as an aside, here is a very handy fact worth knowing:

FACT:   A set of eigenvectors of  the matrix  A  which correspond to distinct eigenvalues of  A  must necessarily be linearly independent.

Nice application:

APPLICATION:  If the n´n  matrix  A  has  n distinct eigenvalues,  then  A  is  diagonalizable.
				
Let's carry out this whole program on the matrix a of the previous section.

s = Eigensystem[a]

d = DiagonalMatrix[s[[1]]]

p = Transpose[s[[2]]]

MatrixForm[a.p]

MatrixForm[p . d]

d - Inverse[p].a.p

a - p.d.Inverse[p]

Finally, let's verify the above equation, say for the fifth power of  A:

p . d.d.d.d.d . Inverse[p]

a.a.a.a.a

Foxes and Chickens???

Yes, foxes and chickens.  Consider the following problem:  a population of foxes and chickens enjoys a predator/prey relationship which is modelled as a linear relationship.  The coefficients that model the relationship are determined experimentally, and one parameter, the kill rate  k (of chickens by foxes ) varies with different populations and locales.  If  Fn and Cn are the fox and chicken populations at the beginning of year n, then the population in the following year is given by 

		Fn+1  = 0.6  Fn  +  0.5  Cn
		Cn+1 =   -k  Fn  +  1.2  Cn
		
Can you attach a meaning to each of the coefficients?   In matrix form we can write the column vector [Fi, Ci]  as  xi and express the formula as

		xi+1  =  A xi,

where the matrix  A  =  {{0.6, 0.5}, {-k, 1.2}}.   Click the following cell:

a = {{0.6, 0.5},{-k, 1.2}}

Now here is the question:  for what values of the kill rate  k  will the fox and chicken populations die out?  For what values will it grow without bound?  Is there a value for which the populations  will stabilize?

Here is where Mathematica really shines !  We recognize this as a discrete linear dynamical system.  From our earlier discussion,  we realize that the growth of the state vector depends on the largest eigenvalue (in absolute value).  If this eigenvalue is greater than one, we expect that some part of the population will grow without bound.  If it is smaller than one  we expect that both populations will die out in time.  If it is equal to one,  we expect that the populations will stabilize.   So let's make a graph of the size of the eigenvalues  (in absolute value) of  the matrix  a  as a function  of the parameter k:

s[k_] := Max[Abs[Eigenvalues[a]]]

Plot[s[k],{k,0,2}]

Now narrow the graph a little and see if you can find a value of k for which the largest eigenvalue is about 1 and test this value on the population  x  below.

x = {100,1000}

k = .5

a.a.a.a.x

a.a.a.a.a.a.a.a.x

a.a.a.a.a.a.a.a.a.a.a.a.x

a.a.a.a.a.a.a.a.a.a.a.a.a.a.a.a.x

Conclusions ???

Exercises

Find the eigenvalues and eigenvectors for the following matrices.  Which are diagonalizable?

æ3  5  7ö
    1)           A = ç0  2  1÷
                         è1  4  6ø

æ1  0  3  5  7ö
                          ç2 -1  7  0  8÷                 (Hint:  use NSolve[] to approximate the eigenvalues
    2)           B = ÷-5  2  4  0 -1ç                  and find each eigenspace separately.)
                          ç 3  2  0 -6 3 ç
                          è 1  0  2 -1 2ø

æ20   5   0ö
    3)           C =  ç -8  32  1 ç                
                          è  1   1  23ø

4)     For each matrix, find the characteristic polynomial P(x), and evaluate it at that matrix.    	(i.e.,  if the characteristic polynomial for the matrix D is  x2 + 2 x - 1, evaluate 
             D2 + 2 D - I2.)