(*********************************************************************** Mathematica-Compatible Notebook This notebook can be used on any computer system with Mathematica 4.0, MathReader 4.0, or any compatible application. The data for the notebook starts with the line containing stars above. To get the notebook into a Mathematica-compatible application, do one of the following: * Save the data starting with the line of stars above into a file with a name ending in .nb, then open the file inside the application; * Copy the data starting with the line of stars above to the clipboard, then use the Paste menu command inside the application. Data for notebooks contains only printable 7-bit ASCII and can be sent directly in email or through ftp in text mode. Newlines can be CR, LF or CRLF (Unix, Macintosh or MS-DOS style). NOTE: If you modify the data for this notebook not in a Mathematica- compatible application, you must delete the line below containing the word CacheID, otherwise Mathematica-compatible applications may try to use invalid cache data. For more information on notebooks and Mathematica-compatible applications, contact Wolfram Research: web: http://www.wolfram.com email: info@wolfram.com phone: +1-217-398-0700 (U.S.) Notebook reader applications are available free of charge from Wolfram Research. ***********************************************************************) (*CacheID: 232*) (*NotebookFileLineBreakTest NotebookFileLineBreakTest*) (*NotebookOptionsPosition[ 48826, 1481]*) (*NotebookOutlinePosition[ 49703, 1514]*) (* CellTagsIndexPosition[ 49659, 1510]*) (*WindowFrame->Normal*) Notebook[{ Cell[CellGroupData[{Cell["Inner Products and Gram-Schmidt ", "Title", ImageRegion->{{-0, 1}, {0, 1}}], Cell[CellGroupData[{Cell["Review of Norms and Inner Products", "Subsection", ImageRegion->{{-0, 1}, {0, 1}}], Cell[TextData[{ "Let's recall what the situation is: inspired by the standard norm, we \ elevate its basic properties into a definition. This definition gives us \ the ability to invent various concepts of \.aa length\.ba of vectors, \ which is the main purpose of norms.\n\nDEFINITION: A ", StyleBox["norm", FontSlant->"Italic"], " on a vector space V is a real valued function ", "\:f2f7\:f2e7", " v", "\:f2f7\:f2e7", " defined for all vectors v in V which has the following properties: \ For all vectors u, v in V and scalars a, \n\t(1) ", "\:f2f7\:f2e7", " u ", "\:f2f7\:f2e7", " ", "\[GreaterEqual]", " 0 with equality if and only if v = 0.\n\t(2) ", "\:f2f7\:f2e7", " a u ", "\:f2f7\:f2e7", " = ", "\:f2bd", "a ", "\:f2bd", " ", "\:f2f7\:f2e7", " u ", "\:f2f7\:f2e7", ".\n\t(3) ", "\:f2f7\:f2e7", " u + v ", "\:f2f7\:f2e7", " ", "\[LessEqual]", " ", "\:f2f7\:f2e7", " u ", "\:f2f7\:f2e7", " + ", "\:f2f7\:f2e7", " v ", "\:f2f7\:f2e7", ".\nA vector space, together with a norm defined on it, is called a ", StyleBox["normed vector space", FontSlant->"Italic"], ".\n\nThe classical examples are the standard (a.k.a. Euclidean ) \ norms:\n\nEXAMPLE 1: V = R", StyleBox["3", FontVariations->{"CompatibilityType"->"Superscript"}], " and \n\t", "\:f2f7\:f2e7", "(x", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", x", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ", x", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], ") ", "\:f2f7\:f2e7", " = Sqrt [ x", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], StyleBox["2", FontVariations->{"CompatibilityType"->"Superscript"}], " + x", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], StyleBox["2", FontVariations->{"CompatibilityType"->"Superscript"}], "+ x", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], StyleBox["2", FontVariations->{"CompatibilityType"->"Superscript"}], " ]\n\nEXAMPLE 2: V = C", StyleBox["2", FontVariations->{"CompatibilityType"->"Superscript"}], " and __ __\n\t", "\:f2f7\:f2e7", "(z", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", z", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ")", "\:f2f7\:f2e7", " = Sqrt [ z", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], " z", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], " + z", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], " z", StyleBox["2 ", FontVariations->{"CompatibilityType"->"Subscript"}], "] \n\nA non-standard example:\n\nEXAMPLE 3: V = R", StyleBox["3", FontVariations->{"CompatibilityType"->"Superscript"}], " and\n\t", "\:f2f7\:f2e7", "(x", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", x", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ", x", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], ")", "\:f2f7\:f2e7", " = ", "\:f2bd", " x", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], " ", "\:f2bd", " + ", "\:f2bd", " x", StyleBox["2 ", FontVariations->{"CompatibilityType"->"Subscript"}], "\:f2bd", " + ", "\:f2bd", " x", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], " ", "\:f2bd", "\n\t\nNorms, by themselves, cannot give us a concept of \.aaangle between \ vectors\.ba which we need for ideas like \.aa parallel\.ba and \ \.aaperpendicular ( = orthogonal)\.ba vectors. For this we need the \ concept of an \.aa inner product\.ba, which is again completely inspired by \ the standard inner products on a vector space. Here is the definition:\n\n\ DEFINITION: An ", StyleBox["inner product", FontSlant->"Italic"], " on a vector space V is a scalar valued function defined for all \ vectors u, v in V which has the following properties: For all vectors \ u, v, w in V and scalars a, ____\n\t(1) = (if \ scalars are the reals, ignore the conjugate)\n\t(2) = + \ \n\t(3) = a \n\t(4) ", "\[GreaterEqual]", " 0 with equality if and only if u = 0.\nA vector space, together with an \ inner product on it, is called an ", StyleBox["inner product space", FontSlant->"Italic"], ".\n\t\nThere are a number of laws that can be deduced from these \ properties, e.g., that fact that \n<0, v> = 0 for any vector ", StyleBox["v", FontSlant->"Italic"], ". See your text for a larger listing. Some of the more important \ outcomes of the definition are as follows:\n(1) We can deduce the ", StyleBox["Cauchy-Schwarz Inequality", FontSlant->"Italic"], ": For all vectors u, v in V,\n\t\t\t", "\:f2bd", "", "\:f2bd", StyleBox["2", FontVariations->{"CompatibilityType"->"Superscript"}], " ", "\[LessEqual]", " .\n(2) We can define a norm, called the ", StyleBox["induced norm", FontSlant->"Italic"], ", in terms of the inner product:\n\t\t\t", "\:f2f7\:f2e7", " u ", "\:f2f7\:f2e7", " = Sqrt [ ]\nAs you might expect, the standard inner products \ induce the standard norms (check this).\n(3) Thanks to the Cauchy-Schwarz \ inequality we can introduce the angle, t, between two vectors u and v by \ the formula\n\t\t\tCos [t] = ", StyleBox["", FontVariations->{"CompatibilityType"->"Superscript"}], "/ ", StyleBox["(", FontVariations->{"CompatibilityType"->"Subscript"}], StyleBox["\:f2f7\:f2e7", FontVariations->{"CompatibilityType"->"Subscript"}], StyleBox[" u ", FontVariations->{"CompatibilityType"->"Subscript"}], StyleBox["\:f2f7\:f2e7", FontVariations->{"CompatibilityType"->"Subscript"}], StyleBox[" . ", FontVariations->{"CompatibilityType"->"Subscript"}], StyleBox["\:f2f7\:f2e7", FontVariations->{"CompatibilityType"->"Subscript"}], StyleBox[" v ", FontVariations->{"CompatibilityType"->"Subscript"}], StyleBox["\:f2f7\:f2e7", FontVariations->{"CompatibilityType"->"Subscript"}], StyleBox[")", FontVariations->{"CompatibilityType"->"Subscript"}], ".\nIn particular, two vectors are ", StyleBox["orthogonal", FontSlant->"Italic"], " iff Cos [t] = 0, i.e., = 0.\n(4) We can calculate the \ projection ", StyleBox["p", FontSlant->"Italic"], " of the vector ", StyleBox["u", FontSlant->"Italic"], " along the vector ", StyleBox["a", FontSlant->"Italic"], " according to the formula\n\t\t\tp = proj", StyleBox["a", FontVariations->{"CompatibilityType"->"Subscript"}], " (u) = (", StyleBox["", FontVariations->{"CompatibilityType"->"Superscript"}], " / ", StyleBox["", FontVariations->{"CompatibilityType"->"Subscript"}], ") a\nThus we get that u = p + (u - p) is the sum of a vector parallel \ to ", StyleBox["a", FontSlant->"Italic"], " and a vector orthogonal to ", StyleBox["a", FontSlant->"Italic"], ".\n\n\t\nThe classical examples are the standard (a.k.a. Euclidean ) inner \ products:\n\nEXAMPLE 4: V = R", StyleBox["3", FontVariations->{"CompatibilityType"->"Superscript"}], " and \n\t<(x", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", x", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ", x", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], ") , (y", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", y", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ", y", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], ")> = (x", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", x", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ", x", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], ") . (y", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", y", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ", y", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], ") = x", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], " y", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], " + x", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], " y", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], " + x", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], " y", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], "\n\nEXAMPLE 5: V = C", StyleBox["2", FontVariations->{"CompatibilityType"->"Superscript"}], " and __ __\n<(z", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", z", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ") , (w", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", w", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ")> = (z", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", z", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ") . (w", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", w", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ") = z", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], " w", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], " + z", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], " w", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], "]\n\nA non-standard example:\n\nEXAMPLE 6: V = R", StyleBox["3", FontVariations->{"CompatibilityType"->"Superscript"}], " and\n\t<(x", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", x", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ", x", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], ") , (y", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", y", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ", y", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], ")> = 4 x", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], " y", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], " + 3 x", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], " y", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], " + 2 x", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], " y", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], "\nThis gives rise to a non-standard norm, namely\n\t", "\:f2f7\:f2e7", " (x", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", x", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ", x", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], ") ", "\:f2f7\:f2e7", " = Sqrt [ 4 x", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], StyleBox["2", FontVariations->{"CompatibilityType"->"Superscript"}], " + 3 x", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], StyleBox["2", FontVariations->{"CompatibilityType"->"Superscript"}], " + 2 x", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], StyleBox["2", FontVariations->{"CompatibilityType"->"Superscript"}], "]\n\nWe'll see another non-standard example in the third section.\n\nHere \ are two more definitions that are inspired by the inner product idea and have \ proved to be useful. They are motivated by a desire to abstract the good \ properties of the standard basis e", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", e", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ", ...., e", StyleBox["n", FontVariations->{"CompatibilityType"->"Subscript"}], " of R", StyleBox["n", FontVariations->{"CompatibilityType"->"Superscript"}], " (or C", StyleBox["n", FontVariations->{"CompatibilityType"->"Superscript"}], ").\n\nDEFINITION: a set of vectors v", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", v", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ", ..., v", StyleBox["n", FontVariations->{"CompatibilityType"->"Subscript"}], " is an ", StyleBox["orthogonal set of vectors", FontSlant->"Italic"], " if \n{"CompatibilityType"->"Subscript"}], ", v", StyleBox["j", FontVariations->{"CompatibilityType"->"Subscript"}], "> = 0 for all i ", "\[NotEqual]", " j. If, in addition, each v", StyleBox["i", FontVariations->{"CompatibilityType"->"Subscript"}], " is normal, i.e., has unit length, then we say that the set of vectors is \ an ", StyleBox["orthonormal set of vectors", FontSlant->"Italic"], ".\n\nNote that the property of being an orthonormal or orthogonal set is a \ property of the whole set of vectors. The standard example of an orthonormal \ set is (what else) the standard basis {e", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", e", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ", ..., e", StyleBox["n", FontVariations->{"CompatibilityType"->"Subscript"}], "}. Here is an orthogonal set of vectors that is not standard (still using \ the standard inner product, however): \n\tv", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], " = (1, 1, 0), v", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], " = (1, -1, 0), v", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], " = (0, 0, 1).\nNotice that this set of vectors is not orthonormal, \ since the first two vectors don't have unit length. This shortcoming is \ easy to fix up provided the vectors are all nonzero: simply divide each \ vector by its length to obtain a unit length vector. In this case we would \ obtain:\n\tu", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], " = (", StyleBox["1", FontVariations->{"CompatibilityType"->"Superscript"}], "/", StyleBox["Sqrt [2]", FontVariations->{"CompatibilityType"->"Subscript"}], ") (1, 1, 0), u", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], " = (", StyleBox["1", FontVariations->{"CompatibilityType"->"Superscript"}], "/", StyleBox["Sqrt [2]", FontVariations->{"CompatibilityType"->"Subscript"}], ") (1, -1, 0), u", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], " = (0, 0, 1).\n\t\nHere is an important fact about orthogonal sets of \ nonzero vectors (notice that automatically includes any orthonormal set of \ vectors):\n\nFACT: Any orthogonal set of nonzero vectors is a linearly \ independent set.\nProof: Suppose a linear combination of the orthogonal set \ v", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", v", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ", ...v", StyleBox["n", FontVariations->{"CompatibilityType"->"Subscript"}], " of nonzero vectors added up to 0:\n\tc", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], " v", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], " + c", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], " v", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], " + ... + c", StyleBox["n", FontVariations->{"CompatibilityType"->"Subscript"}], " v", StyleBox["n", FontVariations->{"CompatibilityType"->"Subscript"}], " = 0.\nTake the inner product of both sides with the vector v", StyleBox["i", FontVariations->{"CompatibilityType"->"Subscript"}], ", use the linearity property of inner products and obtain that\n\t0 = \ {"CompatibilityType"->"Subscript"}], ", 0> = {"CompatibilityType"->"Subscript"}], " v", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], " + c", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], " v", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], " + ... + c", StyleBox["n", FontVariations->{"CompatibilityType"->"Subscript"}], " v", StyleBox["n", FontVariations->{"CompatibilityType"->"Subscript"}], ">\n\t = c", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], " {"CompatibilityType"->"Subscript"}], ", v", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], "> + c", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], " {"CompatibilityType"->"Subscript"}], ", v", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], "> + ... + c", StyleBox["n", FontVariations->{"CompatibilityType"->"Subscript"}], " {"CompatibilityType"->"Subscript"}], ", v", StyleBox["n", FontVariations->{"CompatibilityType"->"Subscript"}], ">\n\t = c", StyleBox["i", FontVariations->{"CompatibilityType"->"Subscript"}], " {"CompatibilityType"->"Subscript"}], ", v", StyleBox["i", FontVariations->{"CompatibilityType"->"Subscript"}], ">\nBut {"CompatibilityType"->"Subscript"}], ", v", StyleBox["i", FontVariations->{"CompatibilityType"->"Subscript"}], "> = ", "\:f2f7\:f2e7", " v", StyleBox["i", FontVariations->{"CompatibilityType"->"Subscript"}], " ", "\:f2f7\:f2e7", StyleBox["2", FontVariations->{"CompatibilityType"->"Superscript"}], " ", "\[NotEqual]", " 0, since v", StyleBox["i", FontVariations->{"CompatibilityType"->"Subscript"}], " is not the zero vector. Hence, we may divide by \n{"CompatibilityType"->"Subscript"}], ", v", StyleBox["i", FontVariations->{"CompatibilityType"->"Subscript"}], "> and obtain c", StyleBox["i", FontVariations->{"CompatibilityType"->"Subscript"}], " = 0. This is true for any index ", StyleBox["i", FontSlant->"Italic"], ", so the linear combination must be trivial. Therefore, the vectors v", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", v", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ", ...., v", StyleBox["n", FontVariations->{"CompatibilityType"->"Subscript"}], " form a linearly independent set. \n\nWhat good are orthonormal sets? \ Here is a nifty application: suppose we are given a vector ", StyleBox["u", FontSlant->"Italic"], " in the vector space V along with a subspace W. The problem is to \ find the vector in W nearest to ", StyleBox["u", FontSlant->"Italic"], ". If you use your 3-D intuition, visualizing V as R", StyleBox["3", FontVariations->{"CompatibilityType"->"Superscript"}], " and W as a plane in R", StyleBox["3", FontVariations->{"CompatibilityType"->"Superscript"}], " through the origin, you will see that what we are really looking for is \ the projection ", StyleBox["p", FontSlant->"Italic"], " of the vector ", StyleBox["u", FontSlant->"Italic"], " into the subspace W. That is, a vector ", StyleBox["p", FontSlant->"Italic"], " in W such that u - p is orthogonal to all vectors in W. Now if \ u", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", ..., u", StyleBox["n", FontVariations->{"CompatibilityType"->"Subscript"}], " is an orthonormal basis of W, this projection is easy to find:\n\n\tp = \ proj", StyleBox["W", FontVariations->{"CompatibilityType"->"Subscript"}], " (u) = {"CompatibilityType"->"Subscript"}], ", u> u", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], " + {"CompatibilityType"->"Subscript"}], ", u> u", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], " + ... + {"CompatibilityType"->"Subscript"}], ", u> u", StyleBox["n", FontVariations->{"CompatibilityType"->"Subscript"}], ".\n\nCheck for yourself that u - p is orthogonal to every u", StyleBox["i", FontVariations->{"CompatibilityType"->"Subscript"}], ". Since every vector in W is a combination of these u", StyleBox["i", FontVariations->{"CompatibilityType"->"Subscript"}], "'s, it follows that u - p is orthogonal to every vector in W. This \ is exactly what we were looking for.\n" }], "Text", ImageRegion->{{-0, 1}, {0, 1}}]}, Open]], Cell[CellGroupData[{Cell["The Gram-Schmidt Algorithm", "Subsection", ImageRegion->{{-0, 1}, {0, 1}}], Cell[TextData[{ "Here is the situation: we have a subspace W of R", StyleBox["3", FontVariations->{"CompatibilityType"->"Superscript"}], " given to us by way of a spanning set.\n\t\t\n\t\tW = lin{v", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", v", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ", v", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], "},\n\nwhere v", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], " = {1, -1, 1}, v", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], " = {1, 2, 1}, v", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], " = {-1, -3, 0}. How can we understand what the subspace looks like?\n\n\t\ For one thing, we can replace the spanning set by a basis of the space. \ This tells us the fewest number of vectors of W needed to generate the \ other vectors of W as linear combinations. Click on the next cells to \ determine pivot columns of the Row Echelon Form of A = {v", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", v", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ", v", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], "}." }], "Text", ImageRegion->{{-0, 1}, {0, 1}}], Cell[CellGroupData[{Cell["v1 = N[{1, -1, 1}]", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["v2 = N[{2, 2, 1}]", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["v3 = N[{4, 0, 3}]", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["a = Transpose[{v1,v2,v3}]", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["MatrixForm[RowReduce[a]]", "Input", ImageRegion->{{-0, 1}, {0, 1}}]}, Open]], Cell[TextData[{ "Now we see that v", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], " and v", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], " form a basis for W. In fact, you can figure out a linear combination of \ v", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], " and v", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], " that give v3 from the Row Echelon Form. See it? Click on the next cell \ for a useful definition:" }], "Text", ImageRegion->{{-0, 1}, {0, 1}}], Cell["\<\ VectorPlot[v_,view_]:= \ Show[Graphics3D[Join[Map[Line[{{0,0,0},#}]&,v], \t\t{PointSize[0.02], Map[Point[#]&,v]}]],Axes->Automatic, \t\tAxesLabel->{\"x\",\"y\",\"z\"}, \t\tViewPoint->view]\ \>", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell[TextData[{ "Now let's get a picture of the vectors v", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", v", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ", v", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], ". Click the next cells:" }], "Text", ImageRegion->{{-0, 1}, {0, 1}}], Cell["VectorPlot[{v1,v2,v3},{5,-5,10}]", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell[TextData[{ "If you're not satisfied with the picture, move the point of your eyeball \ {5, -5, 10} around a bit and reclick the previous cell to see it from \ different angles (don't use {0,0,0}).\n\tRather than work with the basis v", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", v", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], " of W, we would like to work with something like the standard basis e", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", e", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ". In other words, we would rather work with an orthonormal basis of \ vectors. Is this possible? Yes !! Let's use v", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], " for our first vector. But this isn't quite right, since v", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], " does not have unit length. Rather, let's replace v", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], " by the vector \nu", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], " = ", StyleBox["v", FontVariations->{"CompatibilityType"->"Superscript"}], "1 / ", StyleBox["\:f2f7\:f2e7", FontVariations->{"CompatibilityType"->"Subscript"}], StyleBox[" v1", FontVariations->{"CompatibilityType"->"Subscript"}], StyleBox["\:f2f7\:f2e7:", FontVariations->{"CompatibilityType"->"Subscript"}] }], "Text", ImageRegion->{{-0, 1}, {0, 1}}], Cell["u1 = v1 / Sqrt[v1.v1]", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell[TextData[{ "Next, let's replace v", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], " by a vector orthogonal to v", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ". In fact, all we have to do is subtract the projection of v", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], " along u", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], " from the vector v", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], " to obtain a vector orthogonal to u", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ". This won't be a unit length vector, so let's denote the result by u", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], "'. By the way, notice that in the following formula for a projection, we \ don't divide by the factor u", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ".u", StyleBox["1. ", FontVariations->{"CompatibilityType"->"Subscript"}], StyleBox[" Why so?", FontVariations->{"CompatibilityType"->"Subscript"}] }], "Text", ImageRegion->{{-0, 1}, {0, 1}}], Cell["u2p = v2 - (u1.v2) u1", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell[TextData[{ "Now let's turn u", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], "' into a unit vector u", StyleBox["2 by", FontVariations->{"CompatibilityType"->"Subscript"}], StyleBox[" normalizing it.", FontVariations->{"CompatibilityType"->"Subscript"}] }], "Text", ImageRegion->{{-0, 1}, {0, 1}}], Cell["u2 = u2p/Sqrt[u2p.u2p]", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["Finally, let's see what these vectors look like:", "Text", ImageRegion->{{-0, 1}, {0, 1}}], Cell["VectorPlot[{u1,u2},{5,-5,10}]", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["\<\ If your'e not convinced that these vectors are orthogonal, you \ could adjust the viewpoint. Or simply click on the next cell:\ \>", "Text", ImageRegion->{{-0, 1}, {0, 1}}], Cell["u1.u2", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell[TextData[{ "What we have now is an orthonormal basis of W, namely, the vectors u", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], " and u", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ". Consider the following problem: what is the projection of ", StyleBox["u", FontSlant->"Italic"], " into W, where ", StyleBox["u", FontSlant->"Italic"], " is given by the next cell? Now that we have an orthonormal basis of \ W, we can use the nice projection formula given in the previous section. \ That is the next cell. Click it, as well as the cells that verify the \ correctness of our calculations immediately following." }], "Text", ImageRegion->{{-0, 1}, {0, 1}}], Cell["u = {2,1,1}", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["p = (u1.u)*u1 + (u2.u)*u2", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["(u-p).u1", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["(u-p).u2", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell[TextData[{ " O.K., now here's the basic goal of the ", StyleBox["Gram-Schmidt algorithm", FontSlant->"Italic"], ": given a linearly independent set of vectors, v", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", v", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ", .... , v", StyleBox["n", FontVariations->{"CompatibilityType"->"Subscript"}], ", we would like to replace these vectors by an orthonormal set of vectors \ u", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", u", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ", .... , u", StyleBox["n", FontVariations->{"CompatibilityType"->"Subscript"}], ", such that for 1 ", "\[LessEqual]", " i ", "\[LessEqual]", " n, the vectors v", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", v", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ", ...., v", StyleBox["i", FontVariations->{"CompatibilityType"->"Subscript"}], " have exactly the same span as the vectors u", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", u", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ", .... , u", StyleBox["i", FontVariations->{"CompatibilityType"->"Subscript"}], ".\n\nHere's how it is done: Given that we already have built orthonormal \ vectors u", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", u", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ", .... , u", StyleBox["i-1", FontVariations->{"CompatibilityType"->"Subscript"}], " with the desired properties, subtract the projections of v", StyleBox["i", FontVariations->{"CompatibilityType"->"Subscript"}], " in each of the directions u", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", u", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ", ...., u", StyleBox["i-1", FontVariations->{"CompatibilityType"->"Subscript"}], " from v", StyleBox["i", FontVariations->{"CompatibilityType"->"Subscript"}], " and then normalize the resulting vector u", StyleBox["i", FontVariations->{"CompatibilityType"->"Subscript"}], "' by dividing by its length to get the vector u", StyleBox["i", FontVariations->{"CompatibilityType"->"Subscript"}], ". Continue in this way til done.\n\tRemember that the projection of a \ vector ", StyleBox["v", FontSlant->"Italic"], " in the direction of a unit vector ", StyleBox["u", FontSlant->"Italic"], " has the simple formula\n\t\t\t\tproj", StyleBox["u", FontVariations->{"CompatibilityType"->"Subscript"}], "(v) = u\n\nHere are the Gram-Schmidt formulas:\n\t\t\n\t\tu", StyleBox["i", FontVariations->{"CompatibilityType"->"Subscript"}], "' = v", StyleBox["i", FontVariations->{"CompatibilityType"->"Subscript"}], " - {"CompatibilityType"->"Subscript"}], ", v", StyleBox["i", FontVariations->{"CompatibilityType"->"Subscript"}], "> u", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], " - {"CompatibilityType"->"Subscript"}], ", v", StyleBox["i", FontVariations->{"CompatibilityType"->"Subscript"}], "> u", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], " - ... - {"CompatibilityType"->"Subscript"}], ", v", StyleBox["i", FontVariations->{"CompatibilityType"->"Subscript"}], "> u", StyleBox["i-1", FontVariations->{"CompatibilityType"->"Subscript"}], ",\n\t\t\n\t\tui = ", StyleBox["u", FontVariations->{"CompatibilityType"->"Superscript"}], "i", StyleBox["'", FontVariations->{"CompatibilityType"->"Superscript"}], " / ", StyleBox["Sqrt [{"CompatibilityType"->"Subscript"}], StyleBox["i", FontVariations->{"CompatibilityType"->"Subscript"}], StyleBox["', u", FontVariations->{"CompatibilityType"->"Subscript"}], StyleBox["i", FontVariations->{"CompatibilityType"->"Subscript"}], StyleBox["'>]", FontVariations->{"CompatibilityType"->"Subscript"}], "\n\nNow let's see how this works on our v", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", v", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], " (remember, we don't need v", StyleBox["3). Notice that for two vectors, these formulas aren't really \ new. We already did these calculations.", FontVariations->{"CompatibilityType"->"Subscript"}] }], "Text", ImageRegion->{{-0, 1}, {0, 1}}], Cell[CellGroupData[{Cell["up1 = v1", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["u1 = up1 / Sqrt[up1 . up1]", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["up2 = v2 - (u1 . v2) u1", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["u2 = up2 / Sqrt[up2 . up2]", "Input", ImageRegion->{{-0, 1}, {0, 1}}]}, Open]], Cell["VectorPlot[{u1, u2},{-1,-5,10}]", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell[TextData[{ "Now to see how this works on three or more vectors, do the following: go \ back to the definition for v", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", v", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ", v", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], " above and change the vector v", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], " slightly, so that the three are linearly independent. Then click on the \ following cells (you may want to adjust perspective in the last cell): " }], "Text", ImageRegion->{{-0, 1}, {0, 1}}], Cell[CellGroupData[{Cell["up1 = v1", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["u1 = up1 / Sqrt[up1 . up1]", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["up2 = v2 - (u1 . v2) u1", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["u2 = up2 / Sqrt[up2 . up2]", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["up3 = v3 - (u1 . v3) u1 - (u2 . v3) u2", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["u3 = up3 / Sqrt[up3 . up3]", "Input", ImageRegion->{{-0, 1}, {0, 1}}]}, Open]], Cell["VectorPlot[{u1, u2, u3},{5,-5,10}]", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell[TextData[{ " Just for fun, let's calculate the projection of our vector ", StyleBox["u", FontSlant->"Italic"], " from earlier cells into the subspace spanned by the orthonormal set \ u", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", u", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ", u", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], ". What does this give you? " }], "Text", ImageRegion->{{-0, 1}, {0, 1}}], Cell[" u", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["(u1.u)*u1 + (u2.u)*u2 + (u3.u)*u3", "Input", ImageRegion->{{-0, 1}, {0, 1}}]}, Open]], Cell[CellGroupData[{Cell["Non-Standard Inner Products", "Subsection", ImageRegion->{{-0, 1}, {0, 1}}], Cell["\<\ We are going to look at a problem that, on the face of it, has \ nothing to do with inner products or even vector spaces. But analysis of the \ problem will illustrate why it is sometimes useful to develop ideas beyone \ the level of immediately obvious applications. Sometimes, rather abstract \ ideas can have unexpected practical applications. Suppose you work for a manufacturer of calculators, and are involved in the \ design of a new calculator. The problem is this : as one of the \ \.aafeatures\.ba of this calculator, the designers decided that it would be \ nice to have a key which calculated a transcendental function. For the sake \ of argument, we'll take the function to be F[x] = Sin [ Exp [x]]. Of \ course, it would be more likely that the function would be something like \ Exp[x] or Log[x], but for purposes of our graphs, the differences I want \ to highlight for you show up better with this F[x]. Let's suppose that our \ job is to come up with an adequate way of calculating F[x], say with an \ error no worse than .1 (yes, I know the error is too large, but this will \ illustrate what's going on without getting too messy.) Moreover, suppose \ that it will only be necessary to compute F[x] for values between -1 and \ 1. Now your job is to find a simple way to calculate values of this function \ F[x], for -1 \[LessEqual] x \[LessEqual] 1. Polynomials are a natural idea for approximating functions. From a \ designer's point of view they are particularly attractive because they are so \ easy to implement. Given the coefficients of a polynomial, it is easy to \ design a very efficient and compact algorithm for calculating values of the \ polynomial. Such an algorithm, together with the coefficients of the \ polynomial, would fit nicely into a small ROM for the calculator, or could \ even be microcoded into the chip.\ \>", "Text", ImageRegion->{{-0, 1}, {0, 1}}], Cell["\<\ The first idea that ought to occur you comes from Calculus: what \ about Taylor series, say centered at 0, since our interval is centered about \ 0. Mathematica has some nice functions for building polynomials. Let's try \ it. First, make a definition of the function F[x] for Mathematica to work \ on. \ \>", "Text", ImageRegion->{{-0, 1}, {0, 1}}], Cell["F[x_] := N[Sin[Exp[x]]]", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["Now for the Taylor Series of F, say of degree 3", "Text", ImageRegion->{{-0, 1}, {0, 1}}], Cell["Series[F[x],{x,0,3}]", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell[TextData[{ "What's the weird stuff, O[x]? It says that the leftover terms behave no \ worse than a constant times x", StyleBox["4", FontVariations->{"CompatibilityType"->"Superscript"}], ", which is O.K. by the Remainder Formula for Taylor Polynomials. To get \ a sensible form without that last term, do this:" }], "Text", ImageRegion->{{-0, 1}, {0, 1}}], Cell["q = Normal[Series[F[x],{x,0,3}]]", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["Let's plot these functions:", "Text", ImageRegion->{{-0, 1}, {0, 1}}], Cell["Plot[{F[x], q}, {x, -1,1}]", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell[TextData[{ "Not too good... Go to the previous cells for ", StyleBox["q", FontSlant->"Italic"], " and Plot, and change the 3 to a 4 and Enter. Then try a 5. Let's see \ what the error is. It appears the worst error is at 1. So..." }], "Text", ImageRegion->{{-0, 1}, {0, 1}}], Cell["N[Abs[F[1] - (q /. {x -> 1})]]", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell[TextData[{ "There is another way, which is the point of this experiment. Let's adopt \ the point of view that our functions live in the inner product space C[-1,1] \ of continuous functions on the interval [-1, 1] with the integral type inner \ product. Now let's try to find the function in the space of cubic (3rd \ degree) polynomials P", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], " that is closest to our F[x]. We will build the necessary tools by using \ definitions of inner products as in class. In the following, we will assume \ that the independent variable in each function expression is ", StyleBox["x", FontSlant->"Italic"], "." }], "Text", ImageRegion->{{-0, 1}, {0, 1}}], Cell["InProd[f_,g_] := NIntegrate[f g, {x, -1, 1}]", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["Norm[f_] := Sqrt[InProd[f,f]]", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell[TextData[{ "Now use the Gram-Schmidt algorithm to generate orthonormal polynomials p", StyleBox["0", FontVariations->{"CompatibilityType"->"Subscript"}], ", p", StyleBox["1", FontVariations->{"CompatibilityType"->"Subscript"}], ", p", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ", and p", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], " from the polynomials 1, x, x", StyleBox["2", FontVariations->{"CompatibilityType"->"Superscript"}], ", x", StyleBox["3", FontVariations->{"CompatibilityType"->"Superscript"}], ". These polynomials are famous enough to have a name attached to them: \ they are the ", StyleBox["Legendre polynomials", FontSlant->"Italic"], "." }], "Text", ImageRegion->{{-0, 1}, {0, 1}}], Cell["p0 = 1", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["p0 = p0 / Norm[p0]", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["p1 = x - InProd[p0,x] p0", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["p1 = p1 / Norm[p1]", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["p2 = x^2 - InProd[p0,x^2] p0 - InProd[p1,x^2] p1", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["p2 = p2 / Norm[p2]", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["\<\ p3 = x^3 - InProd[p0,x^3] p0 - InProd[p1,x^3] p1 - InProd[p2,x^3] p2\ \>", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["p3 = p3 / Norm[p3]", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell[TextData[{ "Finally, let's calculate the projection of our F into P", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], " and see how it does against F[x]." }], "Text", ImageRegion->{{-0, 1}, {0, 1}}], Cell["p = InProd[p0,F[x]]p0+InProd[p1,F[x]]p1+InProd[p2,F[x]]p2", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell["Plot[{F[x],p},{x,-1,1}]", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell[TextData[{ "This isn't so great. Try adding an extra term so as to calculate the \ projection of F into P", StyleBox["3", FontVariations->{"CompatibilityType"->"Subscript"}], ". Testing, for example, at the endpoints, in the following line confirms \ what the graph suggests." }], "Text", ImageRegion->{{-0, 1}, {0, 1}}], Cell["Abs[F[1] - p /. {x -> 1}]", "Input", ImageRegion->{{-0, 1}, {0, 1}}], Cell[TextData[{ "To summarize: what we have accomplished is to obtain a polynomial of \ degree 3 that does as nearly as good a job as the 5th degree Taylor \ polynomial in approximating F[x], at least as far as the worst error is \ concerned. Could we do better with a third degree? Actually, yes. But we \ would have to use an even weirder inner product : InProd[f_,g_] = \n\ NIntegrate[f g / Sqrt[1 - x^2], {x, -1, 1}]. The resulting orthonormal \ polynomials that would be generated from 1, x, x", StyleBox["2", FontVariations->{"CompatibilityType"->"Superscript"}], ", etc, are also famous enough to have a name: the ", StyleBox["Tschebychev polynomials", FontSlant->"Italic"], ". But that's a story for another course......" }], "Text", ImageRegion->{{-0, 1}, {0, 1}}]}, Open]], Cell[CellGroupData[{Cell["Exercises", "Subsection", ImageRegion->{{-0, 1}, {0, 1}}], Cell["\<\ (1) For each of the following sets of vectors, produce an \ orthonormal set which spans the same space as the original vectors: (a) {(1,1,0), (1,0,1), (1,1,1)} (b) {(0,1,1,1), (1,0,1,1), (1,1,2,2), (1,1,1,0)} (c) {(1,0,1,2,-1),(3,-1,3,7,4),(0,0,0,0,0),(-1,3,2,4,7),(-2,5,6,7,9)}\ \ \>", "Text", ImageRegion->{{-0, 1}, {0, 1}}], Cell[TextData[{ "(2) Define the inner product f *g by f * g = Integrate [f(x) g(x), {x, 0, \ ", "\[Pi]", "}]. The set of \n continuous functions with this inner product is \ an inner product space.\n \n (a) Generate an othogonal basis for \ the span of the set of \.aavectors\.ba\n {1, Exp[x], Exp [2 x], \ Exp [3 x]} (Hint: use N to approximate constants.)\n (b) Now try to \ generate an othogonal basis from the set of vectors\n {1, Sin [x], \ Sin [2 x], Sin [3 x]}. Can you make any further predictions?\n (c) Let \ f[x] = x, for ", StyleBox["-", FontVariations->{"CompatibilityType"->"Superscript"}], StyleBox["\[Pi]", FontVariations->{"CompatibilityType"->"Superscript"}], " / ", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], " ", "\[LessEqual]", " x < ", StyleBox["\[Pi]", FontVariations->{"CompatibilityType"->"Superscript"}], "/", StyleBox["2", FontVariations->{"CompatibilityType"->"Subscript"}], ". Find the projection of f onto each of the vectors\n you \ obtained in (2b). \n The series ", "\[Sum]", " [(f * Sin [n x]) Sin [n x] + (f * Cos [n x]) Cos [n x] ] is called the\n \ ", StyleBox["Fourier Series ", FontSlant->"Italic"], "of", StyleBox[" ", FontSlant->"Italic"], "a function f", StyleBox[". ", FontSlant->"Italic"], " If f[x] = - f[-x] (as is here) then f * Cos [n x] = 0 ", "\[ForAll]", "n." }], "Text", ImageRegion->{{-0, 1}, {0, 1}}]}, Open]]}, Open]] }, FrontEndVersion->"4.0 for X", ScreenRectangle->{{0, 1152}, {0, 864}}, WindowToolbars->{}, CellGrouping->Manual, WindowSize->{520, 600}, WindowMargins->{{316, Automatic}, {Automatic, 132}}, PrivateNotebookOptions->{"ColorPalette"->{RGBColor, -1}}, ShowCellLabel->True, ShowCellTags->False, RenderingOptions->{"ObjectDithering"->True, "RasterDithering"->False}, Magnification->1.25 ] (*********************************************************************** Cached data follows. If you edit this Notebook file directly, not using Mathematica, you must remove the line containing CacheID at the top of the file. The cache data will then be recreated when you save this file from within Mathematica. ***********************************************************************) (*CellTagsOutline CellTagsIndex->{} *) (*CellTagsIndex CellTagsIndex->{} *) (*NotebookFileOutline Notebook[{ Cell[CellGroupData[{ Cell[1739, 51, 83, 1, 70, "Title"], Cell[CellGroupData[{ Cell[1845, 54, 90, 1, 70, "Subsection"], Cell[1938, 57, 21849, 685, 70, "Text"] }, Closed]], Cell[CellGroupData[{ Cell[23819, 744, 82, 1, 70, "Subsection"], Cell[23904, 747, 1389, 37, 70, "Text"], Cell[CellGroupData[{ Cell[25316, 786, 69, 1, 70, "Input"], Cell[25388, 789, 68, 1, 70, "Input"], Cell[25459, 792, 68, 1, 70, "Input"], Cell[25530, 795, 76, 1, 70, "Input"], Cell[25609, 798, 75, 1, 70, "Input"] }, Open ]], Cell[25696, 801, 590, 17, 70, "Text"], Cell[26289, 820, 248, 7, 70, "Input"], Cell[26540, 829, 373, 12, 70, "Text"], Cell[26916, 843, 83, 1, 70, "Input"], Cell[27002, 846, 1583, 39, 70, "Text"], Cell[28588, 887, 72, 1, 70, "Input"], Cell[28663, 890, 1206, 32, 70, "Text"], Cell[29872, 924, 72, 1, 70, "Input"], Cell[29947, 927, 347, 10, 70, "Text"], Cell[30297, 939, 73, 1, 70, "Input"], Cell[30373, 942, 98, 1, 70, "Text"], Cell[30474, 945, 80, 1, 70, "Input"], Cell[30557, 948, 185, 4, 70, "Text"], Cell[30745, 954, 56, 1, 70, "Input"], Cell[30804, 957, 758, 18, 70, "Text"], Cell[31565, 977, 62, 1, 70, "Input"], Cell[31630, 980, 76, 1, 70, "Input"], Cell[31709, 983, 59, 1, 70, "Input"], Cell[31771, 986, 59, 1, 70, "Input"], Cell[31833, 989, 4835, 147, 70, "Text"], Cell[CellGroupData[{ Cell[36691, 1138, 59, 1, 70, "Input"], Cell[36753, 1141, 77, 1, 70, "Input"], Cell[36833, 1144, 74, 1, 70, "Input"], Cell[36910, 1147, 77, 1, 70, "Input"] }, Open ]], Cell[36999, 1150, 82, 1, 70, "Input"], Cell[37084, 1153, 671, 17, 70, "Text"], Cell[CellGroupData[{ Cell[37778, 1172, 59, 1, 70, "Input"], Cell[37840, 1175, 77, 1, 70, "Input"], Cell[37920, 1178, 74, 1, 70, "Input"], Cell[37997, 1181, 77, 1, 70, "Input"], Cell[38077, 1184, 89, 1, 70, "Input"], Cell[38169, 1187, 77, 1, 70, "Input"] }, Open ]], Cell[38258, 1190, 85, 1, 70, "Input"], Cell[38346, 1193, 522, 16, 70, "Text"], Cell[38871, 1211, 53, 1, 70, "Input"], Cell[38927, 1214, 84, 1, 70, "Input"] }, Closed]], Cell[CellGroupData[{ Cell[39043, 1217, 83, 1, 70, "Subsection"], Cell[39129, 1220, 1940, 30, 70, "Text"], Cell[41072, 1252, 365, 7, 70, "Text"], Cell[41440, 1261, 74, 1, 70, "Input"], Cell[41517, 1264, 97, 1, 70, "Text"], Cell[41617, 1267, 71, 1, 70, "Input"], Cell[41691, 1270, 376, 8, 70, "Text"], Cell[42070, 1280, 83, 1, 70, "Input"], Cell[42156, 1283, 77, 1, 70, "Text"], Cell[42236, 1286, 77, 1, 70, "Input"], Cell[42316, 1289, 296, 7, 70, "Text"], Cell[42615, 1298, 81, 1, 70, "Input"], Cell[42699, 1301, 740, 15, 70, "Text"], Cell[43442, 1318, 95, 1, 70, "Input"], Cell[43540, 1321, 80, 1, 70, "Input"], Cell[43623, 1324, 819, 25, 70, "Text"], Cell[44445, 1351, 57, 1, 70, "Input"], Cell[44505, 1354, 69, 1, 70, "Input"], Cell[44577, 1357, 75, 1, 70, "Input"], Cell[44655, 1360, 69, 1, 70, "Input"], Cell[44727, 1363, 99, 1, 70, "Input"], Cell[44829, 1366, 69, 1, 70, "Input"], Cell[44901, 1369, 136, 4, 70, "Input"], Cell[45040, 1375, 69, 1, 70, "Input"], Cell[45112, 1378, 234, 6, 70, "Text"], Cell[45349, 1386, 108, 1, 70, "Input"], Cell[45460, 1389, 74, 1, 70, "Input"], Cell[45537, 1392, 340, 8, 70, "Text"], Cell[45880, 1402, 76, 1, 70, "Input"], Cell[45959, 1405, 800, 15, 70, "Text"] }, Closed]], Cell[CellGroupData[{ Cell[46791, 1422, 65, 1, 70, "Subsection"], Cell[46859, 1425, 380, 10, 70, "Text"], Cell[47242, 1437, 1563, 43, 70, "Text"] }, Closed]] }, Closed]] } ] *) (*********************************************************************** End of Mathematica Notebook file. ***********************************************************************)