Hahn-Banach Theorem

Let $X$ be a normed space and $M$ a linear space. Can we extend a linear functional for $M$ to a linear functional on $X$?
Yes. Let $\{ m_i : i \in I \}$ be a linear basis for $M$ and let $\{ x_j : j \in J \} \subset X$ such that $\{ m_i \} \cup \{ x_j\}$ is a linear basis for $X$. Given $f$, define $F$ by

$$F( \sum_{i \in I} \alpha_i m_i + \sum_{j \in J} \beta_j x_J ) = \sum_{i \in I} \alpha_i f(m_i)$$

So $F\vert _M = f$ and $F$ is a linear functional. We have a problem - if $f$ is bounded, then even if $F$ is bounded, we don't have any bound on the norm of $F$.

Example:

$$y = x_j \alpha m_i$$

where $\vert\alpha\vert large$. $\vert F(y)\vert$ is large, even though $\Vert y\Vert$ is small.

We really want an extension $F$ so that $\Vert F\Vert$ is comparable to $\Vert f\Vert$. In fact, we'll be able to do this quite generally, i.e., for any seminorm.

Lemma 10.1.1   Let $X$ be a vector space over $\mathbb{C}$. There is a one to one correspondence between the $\mathbb{R}$-linear functionals $f:X \rightarrow \mathbb{R}$ and $\mathbb{C}$-linear functionals $g:X \rightarrow \mathbb{C}$ given by $g \mapsto Re(g)$ and $f \mapsto \tilde f$ where

$$\tilde f(x) = f(x) - i f(ix)$$

for all $x \in X$.

Moreover, for a seminorm $p$, for all $x \in X$,

$$\vert f(x)\vert \leq p(x), \, \forall x \in X$$

iff $\vert\tilde f(x)\vert \leq p(x)$ for all $x \in X$. In particular, if $X$ is a normed vector space, $\Vert f\Vert _{X^*} = \Vert\tilde f\Vert _{X^*}$.

Proof. It's easy to see that $\Re g$ is $\mathbb{R}$-linear for a $\mathbb{C}$-linear functional $g:X \rightarrow \mathbb{C}$. To see that $\tilde f$ is $\mathbb{C}$-linear, let $x, y \in X$, $\alpha,\beta \in \mathbb{R}$,

$$\tilde f(x + y) = f(x + y) - i f(i(x+y)) = f(x) - i f(ix) + f(y) - i f(iy) = \tilde f(x) + \tilde f(y)$$

$$\tilde f( (\alpha + i \beta) x) = f ( (\alpha + i \beta) x) - i f... ...pha + i \beta) x ) = \alpha f(x) + \beta f(ix) - i \alpha f(ix) + i \beta f(x)$$

$$= \alpha ( f(x) - i f(ix) ) + i \beta (f(x) - if(ix) ) = (\alpha + i \beta) \tilde f(x)$$

Clearly, $\Re(\tilde f) = f$. It's easy to show $\tilde{ \Re g } = g$.
Fix a seminorm in $p$. If $\vert\tilde f(x)\vert \leq p(x)$, then

$$f(x) = \Re \tilde f(x) \leq \vert\tilde f(x) \vert \leq p(x)$$

$$- f(x) = \Re \tilde f(-x) \leq \vert\tilde f(X) \vert \leq p(x)$$

Thus, $\vert f(x)\vert \leq p(x)$ for all $x$.
Conversely, suppose $\vert f(x)\vert \leq p(x)$ for all $x$. Fix $x \in X$ and pick $\theta \in \mathbb{R}$ such that $\tilde f(x) = e^{i \theta} \vert\tilde f(x) \vert$. Then,

$$\vert\tilde f(x) \vert = \tilde f( e^{-i \theta} x) = \Re \tilde ... ...{-i \theta} x) = \vert f(e^{-i \theta} x) \vert \leq p(e^{-i \theta} x) = p(x)$$

$\qedsymbol$

Definition 10.1.2   Let $X$ be a vector space over $\mathbb{R}$. We call $f:X \rightarrow \mathbb{R}$ a sublinear functional if

1. $f(x + y) \leq f(x) + f(y)$, $\forall x,y \in X$.
2. $f(\alpha x) = \alpha f(x)$, $\forall x \in X$, $\forall \alpha \geq 0$

Every seminorm is sublinear functional but not conversely.

Theorem 10.1.3   Hahn-Banach Theorem
Let $X$ be a vector space over $\mathbb{R}$ and $q$ a sublinear functional. If $M$ is a linear subspace of $X$ and $f:M \rightarrow \mathbb{R}$ is a linear functional with $f(m) \leq q(m) \forall m \in M$, then there is a linear functional $F:X \rightarrow \mathbb{R}$ such that $F(x) \leq q(x) \forall x \in X$ and $F(m) = f(m)$ for all $m \in M$.

Proof. Consider first the special case where $\dim(X/M) = 1$.
Fix $x_0 \in M$ and note

$$X = \{ t x_0 + y : t \in \mathbb{R}, y \in M \}$$

Thus any extension of $F$ is uniquely determined by its value at $x_0$. Let $F$ be an extension of $f$ and let $\alpha_0 = F(x_0)$. For $y \in M$,

$$F(x_0 + y) = \alpha_0 + f(y) \leq q(x_0 + y)$$

Then,

$$\alpha_0 \leq -f(y) + q(x_0 + y)$$

for all $y \in M$. Conversely, if this property holds for all $y \in M$, then, supposing $t > 0$,

$$\alpha_0 \leq \frac1t( -f(ty) + q(tx_0 + ty) )$$

Letting $y_1 = ty \in M$, then

$$t \alpha_0 \leq -f(y_1) + q(t x_0 + y_1).$$

That is,

$$q(t x_0 + y_1 ) \geq t \alpha_0 + f(y_1) = F(t x_0 + y_1)$$

Thus, we have

$$F(t x_0 + y_1) \leq q(t x_0 + y_1)$$

for all $t \geq 0$ and all $y_1 \in M$ (pick $y = \frac{y_1}{t}$).
Similarly,

$$\alpha_0 \geq f(z) - q(-x_0 + z), \, \forall z \in M$$

holds if and only if

$$F(t x_0 + z_1) \leq q(t x_0 + z)$$

for all $z_1 \in M$ and all $t < 0$.

Thus, we will have a suitable extension as long as

$$\sup_{z \in M} f(z) - q(-x_0 + z)$$

is less than or equal to

$$\int_{y \in M} - f(y) + q(x_0 + y)$$

Rearranging, we need

$$f(z + y) \leq q(x_0 + y) + q(-x_0 + z), \, \forall y, z \in M$$

But,

$$f(z+y) \leq q( z+y) = q(x_0 + y - x_0 + z) \leq q(x_0 + y ) + q(-x_0 + z)$$

so we have a suitable choice for $\alpha_0$.

Exercise Find $\alpha_0$ in Monday's example.
$X = \mathbb{R}^2$, $M = \{ (x,0) : x \in \mathbb{R}\}$, $f((x,0)) = x$, $x_1 = (10,1)$, and $q$ is the $\ell^1$ norm.

For the general case, we use Zorn's Lemma. Let $\mathcal{S}= \{ (\mathcal{N}, g) : \mathcal{N}$ linear subspace of $X$ containing $M$ $, g : \mathcal{N}\rightarrow \mathbb{R}$ a linear functional extending $f$ with $g(x) &le#leq;q(x) &forall#forall;x &isin#in;N$$\}.$

Partially order $\mathcal{S}$ by defining $(\mathcal{N}, g) \leq (\mathcal{O}, h)$ if $\mathcal{N}\leq \mathcal{O}$ and $h_\mathcal{N}= g$. (Exercise: this is a partial order).

If we can find $(X,g) \in \mathcal{S})$, then $g$ will satisfy the conclusion of the thoerem.

Friday, November 4, 2005:

Let $\{ \mathcal{N}_i,g_i) : c \in I\}$ be a totally ordered subset of $\mathcal{S}$. Let

$$\mathcal{N}= \bigcup_{i \in I} \mathcal{N}_i$$

Then, $\mathcal{N}$ is a linear subspace. If $x \in \mathcal{N}$, $\alpha \in \mathbb{F}$, then $\exists i \in I$ such that $x \in \mathcal{N}_i$ so $\alpha x \in \mathcal{N}_i$. If $x,y \in \mathcal{N}$, then $\exists i,j \in I$ such that $x \in \mathcal{N}_i, y \in \mathcal{N}_j$. Since the set is totally ordered, $\mathcal{N}_i \leq \mathcal{N}_j$ or $\mathcal{N}_j \leq \mathcal{N}_i$. WLOG, $\mathcal{N}_I \subset \mathcal{N}_j$, then $x + y \in \mathcal{N}_J \subset \mathcal{N}$. Thus $\mathcal{N}$ is a linear subspace. Define $g: \mathcal{N}\rightarrow \mathbb{F}$ by

$$g(x) = g_i(x)$$

if $x \in \mathcal{N}_i$. This is well defined; if $x \in \mathcal{N}_1, x \in \mathcal{N}_j$, as this is a total ordering WLOG, $(\mathcal{N}_i,g_i ) \leq (\mathcal{N}_j,g_j)$. Thus, $g_j\vert _{\mathcal{N}_i} = g_i$. since $x \in \mathcal{N}_i$, $g_j(x) = g_1(x)$. So $g(x)$ is well-defined.

Since all the $g_i$'s extend $f$, $g$ extends $f$. As $g_i(x) \leq q(x)$ for all $x \in \mathcal{N}_i$, we have $g(x) \leq q(x)$ for all $x \in \mathcal{N}$. Further, $g$ is linear (proof is similar to above).

Thus, $(\mathcal{N},g) \in \mathcal{S}$. By construction, $\mathcal{N}_i \subset \mathcal{N}$ and $g\vert _{\mathcal{N}_i} = g_i$, so $(\mathcal{N}_i,g_i) \leq (\mathcal{N},g)$. So $(\mathcal{N},g)$ is a maximal element.

By Zorn, $\mathcal{S}$ has a maximal element, $(Y,F)$.

Claim: $Y = X$. If $Y \neq X$, then by the special case, we can we can extend $F$ to a linear functional on a properly larger linear subspace of $X$. This contradicts the maximality of $(Y,F)$. $\qedsymbol$

Theorem 10.1.4   Hahn-Banach Theorem - seminorm version
Let $X$ be a vector space over $\mathbb{F}$, $p$ a seminorm on $X$, $M$ a linear subspace of $X$. If $f:M \rightarrow \mathbb{F}$ is a linear functional with $\vert f(x)\vert \leq p(x)$ for all $x \in M$, then there is $F:X \rightarrow \mathbb{F}$ with $F(x) = f(x)$ for all $x \in M$ and $\vert F(x)\vert \leq p(x)$ for all $x \in X$.

Proof. If $\mathbb{F}= \mathbb{R}$, then apply the Hahn-Banach theorem to get $F:X \rightarrow \mathbb{R}$ extending $f$ with $F(x) \leq p(x)$ for all $x$. Then,

$$-F(x) = F(-x) \leq p(-x) = p(x)$$

So, $\vert F(x)\vert \leq p(x)$.
If $\mathbb{F}= \mathbb{C}$, then let $g = Re f$. By the Lemma, $\vert g(x)\vert \leq p(x)$ for all $x \in M$. Apply case (1) to $g$ to get $G:X \rightarrow \mathbb{F}$ extending $g$ with $\vert G(x)\vert \leq p(x)$ for all $x \in X$.
Let $F = \tilde G$. Then $F$ extends $f$, and by the Lemma, $\vert F(x)\vert \leq p(x)$. $\qedsymbol$

Corollary 10.1.5   Let $X$ be a normed space over $\mathbb{F}$ and $M$ a linear subspace. Let $f:M \rightarrow \mathbb{F}$ be a linear functional. Then, there is an extension $F:X \rightarrow \mathbb{F}$ with $\Vert F\Vert = \Vert f\Vert$.

Proof. Let $p(x) = \Vert f\Vert \cdot \Vert x\Vert$ in the theorem. $\qedsymbol$

Corollary 10.1.6   Let $X$ be a normed space, $x_1, \ldots, x_n$ linearly independent elements of $X$, $\alpha_1, \ldots, \alpha_n \in \mathbb{F}$. Then there is $F \in X^*$ with $F(x_i) = \alpha_i$ for $1 \leq i \leq d$.

Proof. Define $f$ from $\bigvee\{x_1, \ldots, x_n\}$ into $\mathbb{F}$ by $f(x_i) = \alpha_i$ for all $i$ and extend by linearity. Since $\dim(\bigvee\{x_1, \ldots,x_n\})$ is finite, $f$ is continuous. Thus, by Cor. (1), $f$ has a continuous extension $F:X \rightarrow \mathbb{F}$. $\qedsymbol$

Corollary 10.1.7   For $x \in X$, a normed space

$$\Vert x\Vert = \sup\{ \vert f(x)\vert : f \in X^*, \Vert f\Vert \leq 1 \}.$$

Moreover, this supremum is attained.

Proof. Since $\vert f(x)\vert \leq \Vert f\Vert \cdot \Vert x\Vert$, the RHS of the equation is clearly less than or equal to $\Vert x\Vert$.
To see that the supremum is attained, define $f$ on $\mathbb{F}x$ by $f(\alpha x) = \alpha \Vert x\Vert$ for $\alpha \in \mathbb{F}$. Then, $\Vert f\Vert = 1$. By the first corollory, there exists $F:X \rightarrow \mathbb{F}$ with $\Vert F\Vert = 1$ and $\vert F(x)\vert = \Vert x\Vert$. So, RHS of the equation actually equals $\Vert x\Vert$. $\qedsymbol$

Corollary 10.1.8   For a normed vector space $X$, $M \leq X$, $x_0 \in X/M$ and $d = dist(x_0, M)$, then there is $f \in X^*$ with $\Vert f\Vert = 1$, $f\vert _M \equiv 0$, $f(x_0) = d$.

Proof. Let $Q$ be the quotient map from $X$ to $X/M$. By Corollary (3), there is $g \in (X/M)^*$, $\Vert g\Vert = 1$ so that $g(Qx_0) = \Vert Q x_0\Vert = d$.
Let $f = g \circ Q$. Clearly, $f(x_0) = d$, $f$ is linear, $f\vert _M = 0$, and $\Vert f\Vert \leq 1$. Pick a sequence $y_n \in M$ such that $\Vert x+y_n\Vert \rightarrow \Vert Qx\Vert$. Then, $\Vert f\Vert \cdot \Vert x_0 + y_n\Vert \geq \vert f(x_0 + y_n)\vert$.
As $n \rightarrow \infty$, $\Vert x_0 + y_n\Vert \rightarrow d$ and, for all $n$, $f(x_0 + y_n) = d$.
Thus, taking limits, $\Vert f\Vert \cdot d \geq d$. As $d \neq 0$, $\Vert f\Vert = 1$. $\qedsymbol$

Corollary 10.1.9   For $X$ a normed space and $M$ a linear subspace,

$$\bar M = \bigcap \{ \ker f : f \in X^*, M \subset \ker f \}$$

Proof. Let $N$ be the RHS of this equation. If $f \in X^*$, $M \subset \ker f$, then continuity gives $\bar M \subset \ker f$, so $\bar M \subset N$.
If $x_0 \notin \bar M$, corollary 4 gives $f \in X^*$ with $\ker f \supset M$ and $f(x_0) = 0$. Thus, $N \subset \bar M$. $\qedsymbol$

Corollary 10.1.10   For $X$ a normed space and $M$ a linear subspace, $M$ is dense iff $f \in X^*$, $f\vert _M = 0$ implies $f = 0$.

The proof is an exercise.