Steven R. Dunbar
Department of Mathematics
203 Avery Hall
University of Nebraska-Lincoln
Lincoln, NE 68588-0130
http://www.math.unl.edu
Voice: 402-472-3731
Fax: 402-472-8466

Math 489/Math 889
Stochastic Processes and
Advanced Mathematical Finance
Dunbar, Fall 2010

__________________________________________________________________________

Moment Generating Functions

_______________________________________________________________________

Rating

Mathematically Mature: may contain mathematics beyond calculus with proofs.

_______________________________________________________________________________________________

QuestionofDay

Question of the Day

Give some examples of transform methods in mathematics, science or engineering that you have seen or used and explain why transform methods are useful.

_______________________________________________________________________________________________

Key Concepts

The moment generating function converts problems about probabilities and expectations into problems from calculus about function values and derivatives.
The value of the $n$ th derivative of the moment generating function evaluated at $0$ is the value of the $n$ th moment of $X$ .
The sum of independent normal random variables is again a normal random variable whose mean is the sum of the means, and whose variance is the sum of the variances.

__________________________________________________________________________

Vocabulary

The $n$ th moment of the random variable $X$ is $E [X^{n}] = \int_{x} x^{n} f (x) d x$ (provided this integral converges absolutely.)
The moment generating function $ϕ_{X} (t)$ is defined by $ϕ_{X} (t) = E [e^{t X}] = \{\begin{matrix} \sum_{i} e^{t x_{i}} p (x_{i}) & if X is discrete \\ \int_{x} e^{t x} f (x) d x & if X is continuous \end{matrix}$
for all values $t$ for which the integral converges.

__________________________________________________________________________

Mathematical Ideas

We need some tools to aid in proving theorems about random variables. In this section we develop a tool called the moment generating function which converts problems about probabilities and expectations into problems from calculus about function values and derivatives. Moment generating functions are one of the large class of transforms in mathematics that turn a difficult problem in one domain into a manageable problem in another domain. Other examples are Laplace transforms, Fourier transforms, Z-transforms, generating functions, and even logarithms.

The general method can be expressed schematically in the diagram:

\begin{matrix} X_{i}, E [X_{i}], ℙ [X_{i}] & \to & ϕ_{X} (t) \\ ↓ \\ conclusions & \leftarrow & calculations \end{matrix}

Figure 1:

Block diagram of transform methods.

Expectation of Independent Random Variables

Lemma 1. If $X$ and $Y$ are independent random variables, then for any functions $g$ and $h$ :

E [g (X) h (Y)] = E [g (X)] E [h (Y)]

Proof. To make the proof definite suppose that $X$ and $Y$ are jointly continuous, with joint probability density function $F (x, y)$ . Then:

\begin{array}{l} E [g (X) h (Y)] & = \iint_{(x, y)} g (x) h (y) f (x, y) d x d y \\ = \int_{x} \int_{y} g (x) h (y) f_{X} (x) f_{Y} (y) d x d y \\ = \int_{x} g (x) f_{X} (x) d x \int_{y} h (y) f_{Y} (y) d y \\ = E [g (X)] E [h (Y)] . \end{array}

□

The Moment Generating Function

The moment generating function $ϕ_{X} (t)$ is defined by

ϕ_{X} (t) = E [e^{t X}] = \{\begin{matrix} \sum_{i} e^{t x_{i}} p (x_{i}) & if X is discrete \\ \int_{x} e^{t x} f (x) d x & if X is continuous \end{matrix}

for all values $t$ for which the integral converges.

Example. The degenerate probability distribution has all the probability concentrated at a single point. That is, if $X$ is a degenerate random variable with the degenerate probability distribution, then $X = μ$ with probability $1$ and $X$ is any other value with probability $0$ . That is, the degenerate random variable is a discrete random variable exhibiting certainty of outcome. The moment generating function of the degenerate random variable is particularly simple:

\sum_{x_{i} = μ} e^{x_{i} t} = e^{μ t} .

If the moments of order $k$ exist for $0 \leq k \leq k_{0}$ , then the moment generating function is continuously differentiable up to order $k_{0}$ at $t = 0$ . The moments of $X$ can be generated from $ϕ_{X} (t)$ by repeated differentiation:

\begin{array}{l} ϕ_{X}^{'} & = \frac{d}{d t} E [e^{t X}] \\ = \frac{d}{d t} \int_{x} e^{t x} f_{X} (x) d x \\ = \int_{x} \frac{d}{d t} e^{t x} f_{X} (x) d x \\ = \int_{x} x e^{t x} f_{X} (x) d x \\ = E [X e^{t X}] . \end{array}

Then

ϕ_{X}^{'} (0) = E [X] .

Likewise

\begin{array}{l} ϕ_{X}^{''} (t) & = \frac{d}{d t} ϕ_{X}^{'} (t) \\ = \frac{d}{d t} \int_{x} x e^{t x} f_{X} (x) d x \\ = \int_{x} x \frac{d}{d t} e^{t x} f_{X} (x) d x \\ = \int_{x} x^{2} e^{t x} f_{X} (x) d x \\ = E [X^{2} e^{t X}] . \end{array}

Then

ϕ_{X}^{''} (0) = E [X^{2}] .

Continuing in this way:

ϕ_{X}^{(n)} (0) = E [X^{n}]

In words: the value of the $n$ th derivative of the moment generating function evaluated at $0$ is the value of the $n$ th moment of $X$ .

Theorem 2. If $X$ and $Y$ are independent random variables with moment generating functions $ϕ_{X} (t)$ and $ϕ_{Y} (t)$ respectively, then $ϕ_{X + Y} (t)$ , the moment generating function of $X + Y$ is given by $ϕ_{X} (t) ϕ_{Y} (t)$ . In words, the moment generating function of a sum of independent random variables is the product of the individual moment generating functions.

Proof. Using the lemma on independence above:

\begin{array}{l} ϕ_{X + Y} (t) & = E [e^{t (X + Y)}] \\ = E [e^{t X} e^{t Y}] \\ = E [e^{t X}] E [e^{t Y}] \\ = ϕ_{X} (t) ϕ_{Y} (t) . \end{array}

□

Theorem 3. If the moment generating function is defined in a neighborhood of $t = 0$ then the moment generating function uniquely determines the probability distribution. That is, there is a one-to-one correspondence between the moment generating function and the distribution function of a random variable, when the moment-generating function is defined and finite.

Proof. This proof is too sophisticated for the mathematical level we have now. □

The moment generating function of a normal random variable

Theorem 4. If $Z \sim N (μ, σ^{2})$ , then $ϕ_{Z} (t) = exp (μ t + σ^{2} t^{2} ∕ 2)$ .

Proof.

\begin{array}{l} ϕ_{Z} (t) & = E [e^{t X}] \\ = \frac{1}{\sqrt{2 π σ^{2}}} \int_{- \infty}^{\infty} e^{t x} e^{- {(x - μ)}^{2} ∕ (2 σ^{2})} d x \\ = \frac{1}{\sqrt{2 π σ^{2}}} \int_{- \infty}^{\infty} exp (\frac{- (x^{2} - 2 μ x + μ^{2} - 2 σ^{2} t x)}{2 σ^{2}}) d x \end{array}

Now by the technique of completing the square:

\begin{array}{l} x^{2} - 2 μ x + μ^{2} - 2 σ^{2} t x & = x^{2} - 2 (μ + σ^{2} t) x + μ^{2} \\ = {(x - (μ + σ^{2} t))}^{2} - {(μ + σ^{2} t)}^{2} + μ^{2} \\ = {(x - (μ + σ^{2} t))}^{2} - σ^{4} t^{2} - 2 μ σ^{2} t \end{array}

So returning to the calculation of the m.g.f.

\begin{array}{l} ϕ_{Z} (t) & = \frac{1}{\sqrt{2 π σ^{2}}} \int_{- \infty}^{\infty} exp (\frac{- ({(x - (μ + σ^{2} t))}^{2} - σ^{4} t^{2} - 2 μ σ^{2} t)}{2 σ^{2}}) d x \\ = \frac{1}{\sqrt{2 π σ^{2}}} exp (\frac{σ^{4} t^{2} + 2 μ σ^{2} t}{2 σ^{2}}) \int_{- \infty}^{\infty} exp (\frac{- {(x - (μ + σ^{2} t))}^{2}}{2 σ^{2}}) d x \\ = exp (\frac{σ^{4} t^{2} + 2 μ σ^{2} t}{2 σ^{2}}) \\ = exp (μ t + σ^{2} t^{2} ∕ 2) \end{array}

□

Theorem 5. If $Z_{1} \sim N (μ_{1}, σ_{1}^{2})$ , and $Z_{2} \sim N (μ_{2}, σ_{2}^{2})$ and $Z_{1}$ and $Z_{2}$ are independent, then $Z_{1} + Z_{2} \sim N (μ_{1} + μ_{2}, σ_{1}^{2} + σ_{2}^{2})$ . In words, the sum of independent normal random variables is again a normal random variable whose mean is the sum of the means, and whose variance is the sum of the variances.

Proof. We compute the moment generating function of the sum using our theorem about sums of independent random variables. Then we recognize the result as the moment generating function of the appropriate normal random variable.

\begin{array}{l} ϕ_{Z_{1} + Z_{2}} (t) & = ϕ_{Z_{1}} (t) ϕ_{Z_{2}} (t) \\ = exp (μ_{1} t + σ_{1}^{2} t^{2} ∕ 2) exp (μ_{2} t + σ_{2}^{2} t^{2} ∕ 2) \\ = exp ((μ_{1} + μ_{2}) t + (σ_{1}^{2} + σ_{2}^{2}) t^{2} ∕ 2) \end{array}

□

An alternative visual proof that the sum of independent normal random variables is again a normal random variable using only calculus is available at The Sum of Independent Normal Random Variables is Normal

Sources

This section is adapted from: Introduction to Probability Models, by Sheldon Ross.

_______________________________________________________________________________________________

Problems to Solve

Problems to Work for Understanding

Calculate the moment generating function of a random variable $X$ having a uniform distribution on $[0, 1]$ . Use this to obtain $E [X]$ and $Var [X]$ .
Calculate the moment generating function of a discrete random variable $X$ having a geometric distribution. Use this to obtain $E [X]$ and $Var [X]$ .

__________________________________________________________________________

Reading Suggestion

Reading Suggestion:

References

[1] Sheldon M. Ross. Introduction to Probability Models. Academic Press, 9th edition edition, 2006.

[2] S. R. S. Varadhan. Probability Theory. Number 7 in Courant Lecture Notes. American Mathematical Society, 2001.

__________________________________________________________________________

Links

Outside Readings and Links:

Generating Functions in Virtual Laboratories in Probability
Moment Generating Functions in MathWorld.com

__________________________________________________________________________

I check all the information on each page for correctness and typographical errors. Nevertheless, some errors may occur and I would be grateful if you would alert me to such errors. I make every reasonable effort to present current and accurate information for public use, however I do not guarantee the accuracy or timeliness of information on this website. Your use of the information from this website is strictly voluntary and at your risk.

I have checked the links to external sites for usefulness. Links to external websites are provided as a convenience. I do not endorse, control, monitor, or guarantee the information contained in any external website. I don’t guarantee that the links are active at all times. Use the links here with the same caution as you would all information on the Internet. This website reflects the thoughts, interests and opinions of its author. They do not explicitly represent official positions or policies of my employer.

Information on this website is subject to change without notice.

Steve Dunbar’s Home Page, http://www.math.unl.edu/~sdunbar1

Email to Steve Dunbar, sdunbar1 at unl dot edu

Last modified: Processed from LATEX source on October 22, 2010