Matrix Arithmetic

Introduction

Here is one of the main goals of matrix arithmetic:  How can we interpret a linear system (LS) as a simpler kind of equation, namely, as
					A x  =  b,
where  A  is the coefficient matrix of a L S,  b the right hand side vector, and x the vector of unknowns?
	We need a way to multiply matrices.  Actually, addition and multiplication by scalars is useful, too.  They are simpler too, so we do them first.

Matrix Addition

In order to add two matrices, we insist  that  they be exactly the same size.  Then we will do what we did in Calculus: coordinate by coordinate addition.  Thus

DEFINITION:  If A = [ai,j] and B = [bi,j] are both m´n, then A + B is the  m´n matrix
                        given by
		         A + B = [ai,j + bi,j].

a = {{1,1,1},{1,1,2},{1,1,0}}

b = {{2,-3,1},{2,0,0},{3,2,-5}}

a + b

Note: the matrix of zero entries acts like an identity.  It  is  called the zero m´n matrix.

z = Table[0,{i,3},{j,3}]

a + z

Scalar Multiplication

Scalar multiplication means the multiplying of a matrix by a scalar quantity.  Again, we imitate what was done in Calculus with vectors.  Remember that scalar  means a number.  In our case it could mean a  real or complex number, depending on the context.

DEFINITION:  If s  is a scalar and  A  =  [ai,j] an m´n matrix, then s A is defined to be the
                       m´n matrix given by:
		                   s A  =  [s ai,j].

REMARK:  We denote (-1) A  by  - A, and as  usual we write  A - B  =  A + (- B).

2 a

2*a

-1 a

0*a

3 a - 4 b

Linear Combinations

The last cell of the previous section illustrates another very useful idea in linear algebra:

DEFINITION:  A linear combination of the matrices (vectors)  m1, m2, ..., mr is any 
                        expression of the form
	                       a1 m1  + a2 m2  +  ...  + ar mr,
where a1, a2, ..., ar  are scalars.   If this linear combination sums to the zero matrix (vector), then it is called a  trivial  linear combination.  Otherwise, it is a nontrivial  linear combination.

Thus, we saw that 3 a - 4 b is a nontrivial linear combination of the matrices  a  and  b.
Check the following linear combination for triviality and answer this question:  what does this linear combination really say about c in terms of a  and  b?  Test your conclusion by having Mathematica do the calculation.

c={{1,-3,1/5},{1,-3/5,-6/5},{9/5,1,-4}}

3 a - 4 b + 5 c

Matrix Multiplication

This  is  the least obvious idea of today's  material.  Start by recalling how to ªdotº  two vectors of the same  size:  line up the elements side by side, multiply them together and add the results:

v = {1,2,3,4}

w = {2,-1,1,0}

v.w

w.v

Matrix multiplication is an extension of this idea:  let A  be an m´p matrix and  B a  p´n matrix.  Then A B is the m´n matrix whose  (i,j)th entry is the result of the dot product of the ith row of A and jth column of B.  Notice that these two vectors have the same length p.

a.b

b.a

What could play the part of a ªoneº  with respect to matrix multiplication, if anything?  It turns out  that  there  is such a matrix, which we call the identity matrix.  Of course,  just  like zero matrices, the identity matrix can come in different sizes,  though it  is always a square matrix; that is, it has the same  number of rows and columns:

id = IdentityMatrix[3]

Let's see how it works:

a.id

id.a

c = {{2,3,5,1},{1,2,-1,0},{1,2,1,3}}

id . c

d = Transpose[c]

d . id

Now try this:

x = {x1,x2,x3}

a.x

Recall that two matrices  are equal precisely when they are the same size and corresponding entries of each matrix are equal.  With this out of the way, we arrive at a really basic idea about linear systems:  the linear system (LS) with coefficient  matrix A, right hand side vector b and column of unknowns x = {x1, x2, ... , xn}, is exactly equivalent to the matrix equation
					A . x  =  b
This suggests yet another "big idea."  Why not try to solve (LS) by treating matrices like numbers?  Then we could get the unique solution
					x  =  (1/A) . b
But what is it that plays the part of the "inverse" of a matrix?  We know that the earlier matrix a doesn't give a system with a unique solution, so let's try this example (also from class):

a = {{1,1,1},{2,2,5},{4,6,8}}

b = {4,11,24}

Maybe Mathematica knows something about this ...

ia = Inverse[a]

a . ia

ia . a

Now for the real test:

x = ia . b

LinearSolve[a,b]

It works!!!  Clearly, we are going to have to explore this idea.  Click on the earliest version of a and then click the equation defining  ia.  We have some things  to think about,  right?

Inverses

From class we saw how to build (right) inverses using the ªsuperaugmentedº matrix.  We saw that the m´n matrix A had to have rank m, in order to have a right inverse.  Here is the rest of the story for square matrices:
THEOREM:  For a square n´n matrix A, the following are equivalent (TFAE):
       (1)  A.x = b  has a unique solution x for some right hand side vector b.
       (2)  A.x = b  has a unique solution x for all right hand side vectors b.
       (3)  The Row  Echelon  Form of A is  In.
       (4)  rank (A)  = n.
       (5)  A is a product of elementary matrices.
       (6)  A is invertible.

EXAMPLE.  Use Mathematica to find a formula for the general inverse of a 2´2 matrix by using the next cell and the Inverse command.  Check your result.

Clear[A,a,b,c,d]

A={{a,b},{c,d}}

B=Inverse[A]

Simplify[A.B]

Simplify[B.A]

Transposes

Now about transposes.  We already saw this idea last time in lab:

DEFINITION:   The transpose of the m´n matrix A is the n´m matrix obtained by interchanging the rows and columns of A.  It is denoted by AT.

DEFINITION:  A matrix is  symmetric if it equals  its transpose.

Example:  Verify that the following matrix is symmetric by using Mathematica.

a={{4,2,1},{2,4,2},{1,2,3}}

a - Transpose[a]

For complex matrices, there is another idea that is important :

DEFINITION:  The Hermitian transpose of a matrix A is obtained by conjugating each entry of the transpose of A, and is denoted by A* (or sometimes as AH).

DEFINITION:  A matrix is  Hermitian  (or Hermitian symmetric)  if  it equals its  Hermitian transpose.

EXAMPLE:  Verify that the following matrix is Hermitian by using Mathematica:

a ={{1,I,1+I},{-I,-5,2-I},{1-I,2+I,3}}

a - Conjugate[Transpose[a]]

There are some important properties of transposes and Hermitian transposes that we need to know.  The main properties are summarized in the following theorem.  Most of the parts are rather easy to verify directly from definitions.  The statement about  rank is  not so obvious.

KEY FACT:  Let  A  and B  be matrices and c a (complex) scalar.  Then the following hold  (provided  A  and  B have the suitable sizes  for addition or multiplication, as appropriate.)
 	(1)  (AT) T  =  A    and    (A*) *  =  A.
 	(2)  (A + B)T  =  AT  +  BT   and   (A + B)*  =  A* + B*.
 	(3)  (c A)T  =  c (AT)   and   (c A) * = Conj(c) A*.
 	(4)  (A B)T  =  BT AT   and   (A B)*  =  B* A*.
 	(5)  (A-1)T  =  (AT)-1   and   (A-1)*  =  (A*)-1.
 	(6)  rank (A)  = rank (AT)   and    rank (A)  =  rank (A*).

Exercises

Evaluate the following expressions using Mathematica, letting
             æ1  5  7ö                          æ 1  2  3ö
      A = ç0  2  3÷                 B =   ç-1 0   6÷
             è9  0  1ø                          è1  0   0ø
             
    A +  B
    A B
    B A     (Is  matrix multiplication commutative?)
    5 A - 7 B
    A (B + 3 I3) - (A B + 3 A)  
    A3 - 4 A2 - 58 A - 11  I3     
      
    A-1 B-1
    B-1 A-1
    (A B)-1   (Which of the above two expressions is this equal to?)
    AT 
    (A + i B) *  (Use properties to write this another way, only using AT and BT.  Check your
                         answer using Mathematica.  Hint:  For real matrices  R, RT= R *)