Orthonormal Sets and Bases

Definition 1.4.1   Let $\mathcal{H}$ be a Hilbert space. $\mathcal{S}\subset \mathcal{H}$ is an orthogonal set if $s_1 \perp s-2$ for all $s_1, s_2 \in \mathcal{S}$, $s_1 \neq s_2$. $\mathcal{S}\subset \mathcal{H}$ is orthonormal if, in addition, $\Vert s_1\Vert = 1$ for all $s_1 \in \mathcal{S}$.

A basis for $\mathcal{H}$ is a maximal orthonormal set.

Lemma 1.4.2   Zorn's Lemma:
Let $X$ be a partially ordered, non-empty set. If every totally ordered collection of elements of $X$ has an upper bound in $X$, then there is a maximal element in $X$.

Proposition 1.4.3   If $\mathcal{S}$ is an orthonormal set, then there is a basis containing $\mathcal{S}$.

Proof. Use Zorn's Lemma.
Let $X$ be the set of orthonormal sets in $\mathcal{H}$ containing $\mathcal{S}$. We order elements of $X$ by inclusion; i.e., $x \leq y$ if $x \subset y$. Let $S$ be a totally ordered subset of $X$. Let $y = \bigcup_{s \in S} s$.
Claim: $x \in X$; i.e., $x$ is a orthonormal containing $\mathcal{S}$. Since $s \supset \mathcal{S}$ for all $s \in S$, clearly $x \in \mathcal{S}$. Let $h, k$ be distinct elements of $x$. Well, there is some $s_h \in S$ such that $h \in s_h$ and some $s_k \in S$ such that $k \in s_k$. Since $S$ is totally ordered, WLOG $s_h \subset s_k$.
Thus $h, k$ are both elements of the orthonormal set $s_k$. Thus, $h \perp k$. Similarly, for all $h \in y$, $\Vert h\Vert = 1$. By Zorn's lemma, $X$ has a maximal element, $M$. That is, $M$ is a maximal orthonormal set containing $\mathcal{S}$. $\qedsymbol$

Theorem 1.4.4   Gram-Schmidt
For $\mathcal{H}$ a Hilbert space and $(h_n)$ a linearly independent sequence. Then there is an orthonormal sequence $(e_n)$ such that

$$span \{e_1, \ldots, e_n\} = span\{ h_1, \ldots, h_n \}$$

for all $n$.

Proposition 1.4.5   If $\{e_1, \ldots, e_n\}$ is an orthonormal set, then orthogonal projection onto $span\{e_1, \ldots, e_n\}$, $P$, is given by

$$P(h) = \sum_{k=1}^n \left\langle{ h,e_k }\right\rangle e_k.$$

Proof. If $g = \sum_{k=1}^n \left\langle{ h, e_k }\right\rangle e_k$, then $g - h$ is in $(span \{ e_1, \ldots, e_k \})^\perp$ and this is the condition that defines the projection of $h$ onto $span\{e_1, \ldots, e_n\}$. $\qedsymbol$

Theorem 1.4.6   Bessel's Inequality
If $(e_n)$ is an orthonormal sequence and $h \in \mathcal{H}$,

$$\sum_{n=1}^\infty \vert \left\langle{ h, e_n }\right\rangle \vert^2 \leq \Vert h\Vert^2$$

Proof. Letting $g = \sum_{k=1}^n \left\langle{ h, e_k }\right\rangle e_k$, then $\left\langle{ h-g, e_k }\right\rangle = 0$ for $k =1, \ldots, n$ and since $g \in span\{e_1, \ldots, e_n \}$

$$\Vert h\Vert^2 = \Vert h-g\Vert^2 + \Vert g\Vert^2 \geq \Vert g\Vert^2 = \sum_{k=1}^n \vert\left\langle{ h,e_k }\right\rangle \vert^2$$

Thus,

$$\Vert h\Vert^2 \geq \sum_{k=1}^n \vert\left\langle{ h,e_k }\right\rangle \vert^2$$

$\qedsymbol$

Corollary 1.4.7   If $\epsilon$ is an orthonormal set and $h \in \mathcal{H}$, then

$$\{ e \in \epsilon : \left\langle{ h,e }\right\rangle \neq 0 \}$$

is countable

Proof. Fix $n \in \mathbb{N}$. Let $\epsilon _n = \{ e \in \epsilon : \vert \left\langle{ h,e }\right\rangle \vert \geq \frac1n \}$. If $\epsilon _n$ is infinite, we can find a countable subset $\{ e_1, e_2, \ldots \} \subset \epsilon _n$. Thus, by Bessel's inequality,

$$\infty > \Vert h\Vert^2 \geq \sum_{k=1}^\infty \vert\left\langle{ h, e_k }\right\rangle \vert^2 = \infty$$

So $\epsilon _n$ is finite. But,

$$\{ e \in \epsilon : \vert \left\langle{ h,e }\right\rangle \vert > 0 \} = \bigcup_{n \in \mathbb{N}} \epsilon _n$$

so this set is countable. $\qedsymbol$

Definition 1.4.8   Let $I$ be an index set and $\{h_i : i \in I\}$ a collection of vectors in a Hilbert space. Let $\mathcal{F}$ be the collection of all finite subsets of $I$. For each $f \in \mathcal{F}$, define

$$h_f = \sum_{i \in f} h_i$$

Order $\mathcal{F}$ by includsion. $\mathcal{F}$ is a directed set with this partial order, that is, if $f, g \in \mathcal{F}$, then there is a $k \in \mathcal{F}$ with $f \leq k$, $g \leq k$. We say $\sum \{ h_i : i \in I \}$ converges to $h \in \mathcal{H}$ if $\{ h_f : f \in \mathcal{F}\}$ converges as a net. That is, for all $\epsilon > 0$, there is $F \in \mathcal{F}$ such that $\forall G \in \mathcal{F}$ with $G \geq F$, $\Vert h_G - h \Vert \leq \epsilon$.

The previous proposition imples that

$$\sum \{ \vert\left\langle{ h, e }\right\rangle \vert^2 : e \in \epsilon \}$$

converges to a number less than $\Vert h\Vert^2$ for any orthonormal set.

Lemma 1.4.9   If $\epsilon$ is an orthonormal set in $\mathcal{H}$, $h \in \mathcal{H}$,

$$\sum \{ \left\langle{ e, h }\right\rangle e : e \in \epsilon \}$$

converges in $\mathcal{H}$.

Proof. Let $\delta > 0$. We know $\{ e \in \epsilon : \left\langle{ h, e }\right\rangle \neq 0 \}$ is countable, so enumerate it as $e_1, e_2, \ldots$. Since

$$\sum_{i=1}^\infty \vert \left\langle{ h, e_i }\right\rangle \vert^2 \leq \Vert h\Vert^2 < \infty$$

there is $N$ such that

$$\sum_{i=N}^\infty \vert \left\langle{ h,e_i }\right\rangle \vert^2 < \delta$$

Let $F = \{ e_1, \ldots, e_{N-1} \}$. If $A,B \in \epsilon$ are finite subsets and if $F \subset A, F \subset B$, then

$$\left\Vert \sum_{e \in A} \left\langle{ h,e }\right\rangle e - \s... ...rt \sum_{e \in A \triangle B} \left\langle{ h,e }\right\rangle e \right\Vert^2$$

$$\leq \sum_{e \in A \triangle B} \vert\left\langle{ h,e }\right\ra... ...leq \sum_{i=N}^\infty \vert\left\langle{ h,e_i }\right\rangle \vert^2 < \delta$$

Thus, $\sum \{ \left\langle{ h,e }\right\rangle e : e \in \epsilon \}$ is Cauchy in the Hilbert space. Hence, it converges to some vector in $\mathcal{H}$. In fact, it converges to

$$\sum_{i=1}^\infty \left\langle{ h, e_i }\right\rangle e_i$$

$\qedsymbol$

Theorem 1.4.10   Let $\epsilon$ be an orthonormal set in $H$. The following are equivalent:

(a)

$\epsilon$ is a basis.

(b)

$\epsilon ^\perp = 0$.

(c)

$\vee \epsilon = H$.

(d)

For $h \in \mathcal{H}$, $h = \sum \{ \left\langle{ h,e }\right\rangle e : e \in \epsilon \}$.

(e)

For $g,h \in \mathcal{H}$, $\left\langle{ g,h }\right\rangle = \sum \{ \left\langle{ g,e }\right\rangle \left\langle{ e,h }\right\rangle : e \in \epsilon \}$.

(f)

For $h \in \mathcal{H}$, $\Vert h\Vert^2 = \sum \{ \vert \left\langle{ h,e }\right\rangle \vert^2 : e \in \epsilon \}$.

Proof. We break this down into several parts:

(a) $\Rightarrow$ (b)

If $h \in \epsilon ^\perp$ and $h \neq 0$, then $\epsilon \cup \displaystyle \left\{ \frac{h}{\Vert h\Vert} \right\}$ is an orthonormal set properly containing $\epsilon$.

(b) $\Leftrightarrow$ (c)

Already done.

(b) $\Rightarrow$ (d)

Let

$$x = h - \sum \{ \left\langle{ h,e }\right\rangle e : e \in \epsilon \}$$

This is well defined, by the lemma. Let $f \in \epsilon$.

$$\left\langle{ x, f }\right\rangle = \left\langle{ h,f }\right\ran... ...um \{ \left\langle{ h,e }\right\rangle e : e \in \epsilon \}, f }\right\rangle$$

$$= \left\langle{ h,f }\right\rangle - \sum \{ \left\langle{ h,e }\... ...on \} = \left\langle{ h,f }\right\rangle - \left\langle{ h,f }\right\rangle = 0$$

Thus, $(d)$ holds.

(d) $\Rightarrow$ (e)

We've just established (e) in the spacial case $g \in \epsilon$. Thus, (e) holds for $g$ a finite linear combination of elements of $\epsilon$. By (d) every element of $g$ is a limit of such linear combinations.

(e) $\Rightarrow$ (f)

Let $g = h$ in (e).

(f) $\Rightarrow$ (a)

If $\epsilon$ is not a basis, $\exists f$, $\Vert f\Vert = 1$, $f \in \epsilon ^\perp$. Then $\Vert f\Vert^2 \neq 0$, but

$$\sum \{ \vert \left\langle{ f,e }\right\rangle \vert^2: e \in \epsilon \} = 0$$

so (f) is false.

$\qedsymbol$

Friday, 9-2-05:

Proposition 1.4.11   Any two bases in a Hilbert space have the same cardinality.

Proof. Let $A,B$ be two bases and $\eta, \zeta$ their cardinalities. If $\eta$ or $\zeta$ is finite, then the other is also finite, and they are equal (linear algebra / row reduction).

Assume $\eta, \zeta$ are both infinite. For $e \in A$, let

$$B_e = \{ f \in B : \left\langle{ e,f }\right\rangle \neq 0 \}$$

By previous work, $B_e$ is countable. Since $A$ is a basis, $A^\perp = 0$ and so for every $f \in B$, there is at least one little $e \in A$ such that $\left\langle{ f,e }\right\rangle \neq 0$. That is, $$\bigcup_{e \in A} B_e = B$$.

$$\zeta = card(B) \leq card( A \times B_e ) = card( E)$$

Since $B_e$ is countable, this last cardinality $card(A)$. Thus, $\zeta \leq \eta$. Similarly, $\eta \leq \zeta$. By Schroeder-Bernstein, $\eta = \zeta$. $\qedsymbol$

Definition 1.4.12   The dimension of a Hilbert space is the cardinality of a basis.

Recall We say a metric space is separable if it has a countable dense subset.

Proposition 1.4.13   A Hilbert space is separable if and only if its dimension is at most countable.

Proof.

$\Leftarrow$

Let $\mathcal{E}$ be a countable basis. Let

$$X = \{ \sum_{i=1}^n \alpha_i x_i : x_i \in \mathcal{E}, \alpha_i ... ...R}), \alpha_i = p_i + q_i i, p_i,q_i \in \mathbb{Q}(\mathbb{F}= \mathbb{C}) \}$$

Then $X$ is countable. We claim that $X$ is dense. Let $h \in \mathcal{H}$ and let $\epsilon > 0$. Well, $h = \sum_{i=1}^\infty \beta_i e_i$, where $\mathcal{E}= \{ e_1, \ldots \}$ and $\beta_i \in \mathbb{F}$. By Parseval's identity,

$$\sum_{i=1}^\infty \vert\beta_i\vert^2 = \Vert h\Vert^2 < \infty$$

Choose $N$ such that

$$\sum_{N+1}^\infty \vert\beta_i\vert^2 < \frac{\epsilon ^2}4.$$

For $i = 1, \ldots, N$, choose $\alpha_i$ in $\mathbb{Q}$ if $\mathbb{F}= \mathbb{R}$ or in $\mathbb{Q}+ i \mathbb{Q}$ if $\mathbb{F}= \mathbb{C}$ so that $\vert\alpha_i - \beta_i\vert < \frac{\epsilon }{2N}$. Then,

$$\Vert h - \sum_{i=1}^N \alpha_i e_i \Vert \leq \Vert h - \sum_{i=... ...\Vert \sum_{i=1}^N \beta_i e_i - \sum_{i=1}^N \alpha_i e_i \Vert \leq \epsilon$$

So $X$ is dense

$\Rightarrow$

Fact If $(X,d)$ is a separable metric space and $\{B_i = B(x_i,\epsilon _i) : i \in I \}$ are pairwise disjoint, then $I$ is countable.

Proof. Let $D$ be the countable dense set. For each $i \in I$, $D \cap B_i \neq \emptyset$ so we can pick $d_i \in D \cap B_i$. For $i \neq j$, $d_i \neq d_j$, so we have an injection $i \mapsto d_i$ from $I$ into $D$. Thus, $card(I)$ is at most countable. $\qedsymbol$

Consider

$$\{ B(e, 2^{-\frac12}): e \in \mathcal{E}\}$$

If $g \in B(e,2^{-\frac12})$ and $h \in B(f,2^{-\frac12})$ for $e,f \in \mathcal{E}$, $e \neq f$,

$$\Vert g - h\Vert \geq \Vert e - f\Vert - \Vert e - g\Vert - \Vert f - h\Vert > \sqrt2 - \frac1{\sqrt2} - \frac1{\sqrt2} = 0$$

By the fact, $\mathcal{E}$ is countable.

$\qedsymbol$

Definition 1.4.14   Let $\mathcal{H}, K$ be Hilbert spaces. An isomorphism $U: H \rightarrow K$ is a linear surjection such that

$$\left\langle{ Uf, Ug }\right\rangle = \left\langle{ f,g }\right\rangle , \forall f,g \in \mathcal{H}$$

Warning such maps are often called unitary operators. Our text reserves this latter term for the case $H = K$.

Remarks:

1. $U$ is injective - $Uf = 0 \implies \left\langle{ Uf,Uf }\right\rangle = 0 \implies \left\langle{ f,f }\right\rangle = 0 \implies f = 0$.
2. $U^{-1}$ is an isomorphism.
3. $U$ maps Cauchy sequences to Cauchy sequences.

Wednesday, 9-7-05: