Steven R. Dunbar
Department of Mathematics
203 Avery Hall
University of Nebraska-Lincoln
Lincoln, NE 68588-0130
http://www.math.unl.edu
Voice: 402-472-3731
Fax: 402-472-8466

Topics in
Probability Theory and Stochastic Processes
Steven R. Dunbar

__________________________________________________________________________

Analytic Proof of the DeMoivre-Laplace Central Limit Theorem

_______________________________________________________________________

Note: These pages are prepared with MathJax. MathJax is an open source JavaScript display engine for mathematics that works in all browsers. See http://mathjax.org for details on supported browsers, accessibility, copy-and-paste, and other features.

_______________________________________________________________________________________________

Rating

Rating

Mathematicians Only: prolonged scenes of intense rigor.

_______________________________________________________________________________________________

Section Starter Question

Section Starter Question

Derive the Fourier transform of the rectangle function Π(t), where

Π(t) = 0 t > 12 12t = 12 1 t < 12 .

_________________________________________________________________________________

Key Concepts

Key Concepts

  1. Let X be a random variable with probability density function f. The Fourier transform, also known as the characteristic function of f (and also of X) is
    ϕ(ξ) =eiξxf(x)dx.

  2. Let ϕ(ξ) be the Fourier transform (also known as the characteristic function) of a probability distribution such that ϕ(ξ) dξ < . Then there is a bounded continuous density f(x) given by
    f(x) = 1 2πeiξxϕ(ξ)dξ.

  3. For every real ξ,
    lim ncos ξ nn = eξ22.

  4. μ ω1 < r1(t) + r2(t) + + rn(t) n < ω2 = 1 2πeiω2ξ eiω1ξ iξ eξ22 dξ = 1 2πω1ω2 ey22dy.

__________________________________________________________________________

Vocabulary

Vocabulary

  1. The indicator function for the interval [ω1,ω2] is
    g(x) = 1ω1 < x < ω2 0 otherwise .

  2. Let X be a random variable with probability density function f. The Fourier transform, also known as the characteristic function of f (and also of X) is
    ϕ(ξ) =eiξxf(x)dx.

  3. Let f(x) and g(x) be two probability density functions with Fourier transforms ϕ(ξ) and ψ(ξ) respectively. Parseval’s relation is
    eiξtϕ(ξ)g(ξ)dξ =ψ(x t)f(x)dx.

  4. Fourier’s integral formula is
    g(x) = 1 2πg(y)eiξ(xy)dydξ.

__________________________________________________________________________

Mathematical Ideas

Mathematical Ideas

Fourier Transforms and Characteristic Functions

Definition. Let X be a random variable with probability density function f. The Fourier transform, also known as the characteristic function of f (and also of X) is

ϕ(ξ) =eiξxf(x)dx.

Example. The uniform random variable with density function f(x) = 1 2a on [a,a] has Fourier transform

ϕ(ξ) =eiξxf(x)dx =aaeiξx 2a dx = eiξx 2a iξaa = eiξa 2a iξ eiξa 2a iξ = sin(aξ) aξ .

The Fourier transform has many properties, but the subsequent proofs only need one, the Fourier inversion formula:

f(x) = 1 2πeiξxϕ(ξ)dξ.

(valid if ϕ(ξ) dξ < ).

Lemma 1 (Parseval relation). Let f(x) and g(x) be two probability density functions with Fourier transforms ϕ(ξ) and ψ(ξ) respectively. Then

eiξtϕ(ξ)g(ξ)dξ =ψ(x t)f(x)dx.

Proof.

  1. Start with the definition of ϕ(ξ) and multiply by eiξt:
    eiξtϕ(ξ) =eiξ(xt)f(x)dx.

  2. Integrate both sides with respect to g(ξ)dξ to get
    eiξtϕ(ξ)g(ξ)dξ =eiξ(xt)f(x)dxg(ξ)dξ.

  3. Interchange the order of integration and apply the definition of ψ(ξ)
    eiξtϕ(ξ)g(ξ)dξ =ψ(x t)f(x)dx.

Remark. The proof is easily generalized to cumulative distribution functions F(x) and G(x) with associated probability measures dF and dG. See [3].

Theorem 2. Let ϕ(ξ) be the Fourier transform (also known as the characteristic function) of a probability distribution such that ϕ(ξ) dξ < . Then there is a bounded continuous density f(x) given by

f(x) = 1 2πeiξxϕ(ξ)dξ.

Proof.

  1. Consider na(x) = an(ax), the scaled normal density function, where n(x) = 1 2πex22 is the standard normal probability density.
  2. The Fourier transform (characteristic function) of the scaled normal density function is ψa(ξ) = 2πn(ξa), see the Problems for a proof.
  3. Then consider the Parseval relation applied to na(x):
    eiξtϕ(ξ) a 2πea2ξ22dξ =2πn x t a f(x)dx.

    Rearranging the constants and using the evenness of n()

    1 2πeiξtϕ(ξ)ea2ξ22dξ =1 an t x a f(x)dx. (1)
  4. Let
    fa(x) =1 an t x a f(x)dx (2)

    so that fa(x) is the probability density that is the convolution of f with the scaled normal density, fa = n1a f.

  5. From (2), lim a0fa(x) = f(x).
  6. From (1)

    lim a01 an t x a f(x)dx = lim a0 1 2πeiξtϕ(ξ)ea2ξ22dξ = 1 2πeiξtϕ(ξ)dξ.
  7. Using these limits in (1)
    f(x) = 1 2πeiξxϕ(ξ)dξ.

Remark. This proof is adapted from Feller [3, pages 507-511]. This is one of several proofs possible for the inversion theorem. For alternative proofs see [1, pages 177-178], [2, page 155], or [5, pages 20-21].

The Main Idea of the Proof

Let

g(x) = 1ω1 < x < ω2 0 otherwise (3)

Lemma 3 (Fourier’s Formula). As a Fourier integral

g(x) = 1 2πeiω2ξ eiω1ξ iξ eixξdξ. (4)

with the slight exception that g(ω1) = 1 2 and g(ω2) = 1 2.

Proof. Left as an exercise. □

Using the analytic expression for the probability of the sum of Bernoulli trials

μ ω1 < r1(t) + r2(t) + + rn(t) n < ω2 =01g r1(t) + r2(t) + + rn(t) n dt =01 1 2πeiω2ξ eiω1ξ iξ × exp iξr1(t) + r2(t) + + rn(t) n dξdt.

Changing the exponential into a product of cosines comes from equations (2) and (3) in Analytic Model of Coin Flipping.

Interchange the order of integration, justified since r1(t) + r2(t) + + rn(t) assumes only a finite set of values.

μ ω1 < r1(t) + r2(t) + + rn(t) n < ω2 = 1 2πeiω2ξ eiω1ξ iξ 01 exp iξr1(t) + r2(t) + + rn(t) n dtdx = 1 2πeiω2ξ eiω1ξ iξ cos ξ nndξ.

Lemma 4. For every real ξ,

lim ncos ξ nn = eξ22

Proof. This follows from

cos ξ n = 1 ξ2 2n + o 1 n

so that

lim ncos ξ nn = lim n1 ξ2 2n + o 1 nn = eξ22.

Alternatively, taking logarithms and using L’Hospital’s Rule

lim nn log cos ξ n = lim n 1 cos ξ n sin ξ n ξ 2n32 1 n2 = lim n 1 cos ξ n sin ξ n n ξ ξ2 2 = ξ22

Using this limit

lim nμ ω1 < r1(t) + r2(t) + + rn(t) n < ω2 = 1 2πeiω2ξ eiω1ξ iξ eξ22 dξ = 1 2πω1ω2 ey22dy

The interchange of the operations of integration and taking the limit as n needs justification. However, the limits of integration are and + and the function

eiω2ξ eiω1ξ iξ

is not absolutely integrable. Recall that a function is integrable on E if Ef(x)dx < . A function is absolutely integrable on E if E f(x) dx < . A theorem from Lebesgue integration theory says that a function f(x) is integrable if and only if f(x) is absolutely integrable. The interchange of the operations of integration and taking the limit as n is justified if the integrand is integrable, hence absolutely integrable. However, the function

eiωξ iξ = cos(ωξ) + i sin(ωξ) iξ

is not absolutely integrable on (,), see the exercises. Therefore, although this proof of the Central Limit Theorem is fairly direct with Fourier transforms, it is not rigorously justified.

Rigorous justification

The equation (4) is a special case of Fourier’s general formula

g(x) = 1 2πg(y)eiξ(xy)dydx

applied to the indicator function (3). To make the proof above rigorous, introduce two auxiliary functions g𝜖(x) and g𝜖+(x), both graphed in Figure 1.


Functions approximating the indicator function

Figure 1: Auxiliary functions approximating the indicator function. The indicator function is blue with the jump discontinuities drawn as vertical lines.

By construction, g𝜖(x) < g(x) < g 𝜖+(x) and then

01g 𝜖r1(t) + r2(t) + + rn(t) n dt μ ω1 < r1(t) + r2(t) + + rn(t) n < ω2 01g 𝜖+ r1(t) + r2(t) + + rn(t) n dt.

Now

G𝜖(ξ) =g 𝜖(y)eiyξdy and G 𝜖+(ξ) =g 𝜖+(y)eiyξdy

are absolutely integrable as functions of ξ on (,). (See the exercises for a proof.) Now the same argument as above yields rigorously

lim n01g 𝜖r1(t) + r2(t) + + rn(t) n dt = 1 2πey22g 𝜖(y)eiyξdydξ = 1 2πg 𝜖(y)ey22dy

and

lim n01g 𝜖+ r1(t) + r2(t) + + rn(t) n dt = 1 2πey22g 𝜖+(y)eiyξdydξ = 1 2πg 𝜖+(y)ey22dy.

Combining the two previous expressions with the inequality for g𝜖 and g𝜖+

1 2πg 𝜖(y)ey22dy liminf nμ ω1 < r1(t) + r2(t) + + rn(t) n < ω2 limsup nμ ω1 < r1(t) + r2(t) + + rn(t) n < ω2 1 2πg 𝜖(y)ey22dy.

Since the inequality holds for every 𝜖 > 0, it follows that

lim nμ ω1 < r1(t) + r2(t) + + rn(t) n < ω2 = 1 2πg(y)ey22dy = 1 2πω1ω2 ey22dy.

Section Starter Question

Section Ending Answer

x[Π(x)](ω) =eiωxΠ(x)dx = sin x x = sinc(x).

Sources

The definition of the Fourier transform and the inversion Theorem are adapted from William Feller, Introduction to Probability Theory and It Applications, Volume II, Second Edition, by W. Feller, J. Wiley and Sons, pages 507-511. The main results of the section are adapted from Statistical Independence in Probability, Analysis, and Number Theory by Mark Kac, Carus Mathematical Monographs, Number 12, Chapter 3, Sections 1 and 2, pages 36-41. [4]

_______________________________________________________________________________________________

Algorithms, Scripts, Simulations

Algorithms, Scripts, Simulations

Algorithm

Scripts

__________________________________________________________________________

Problems to Work

Problems to Work for Understanding

  1. Show that the Fourier transform (characteristic function) of the scaled normal density function an(ax), where n(x) = 1 2πex22, is ψ(ξ) = 2πn(xa).
  2. Show the function
    eiωξ iξ = cos(ωξ) + i sin(ωξ) iξ

    is not absolutely integrable on (,).

  3. Prove Fourier’s Formula
    g(x) = 1 2πeiω2ξ eiω1ξ iξ eixξdξ.

  4. Show that the function x[g](ξ) is absolutely integrable on (,).

__________________________________________________________________________

Books

Reading Suggestion:

References

[1]   Leo Breiman. Probability. Addison Wesley, 1968.

[2]   Kai Lai Chung. A Course in Probability Theory. Academic Press, 1974.

[3]   William Feller. A Introduction to Probability Theory and It Applications, Volume II, Second Edition, volume II. John Wiley and Sons, second edition, 1971.

[4]   Mark Kac. Statistical Independence in Probability, Analysis and Number Theory, volume 12 of The Carus Mathematical Monographs. Mathematical Association of America, 1959.

[5]   S. R. S. Varadhan. Probability Theory. Number 7 in Courant Lecture Notes. American Mathematical Society, 2001.

__________________________________________________________________________

Links

Outside Readings and Links:

__________________________________________________________________________

I check all the information on each page for correctness and typographical errors. Nevertheless, some errors may occur and I would be grateful if you would alert me to such errors. I make every reasonable effort to present current and accurate information for public use, however I do not guarantee the accuracy or timeliness of information on this website. Your use of the information from this website is strictly voluntary and at your risk.

I have checked the links to external sites for usefulness. Links to external websites are provided as a convenience. I do not endorse, control, monitor, or guarantee the information contained in any external website. I don’t guarantee that the links are active at all times. Use the links here with the same caution as you would all information on the Internet. This website reflects the thoughts, interests and opinions of its author. They do not explicitly represent official positions or policies of my employer.

Information on this website is subject to change without notice.

Steve Dunbar’s Home Page, http://www.math.unl.edu/~sdunbar1

Email to Steve Dunbar, sdunbar1 at unl dot edu

Last modified: Processed from LATEX source on October 12, 2018