Steven R. Dunbar
Department of Mathematics
203 Avery Hall
University of Nebraska-Lincoln
Lincoln, NE 68588-0130
http://www.math.unl.edu
Voice: 402-472-3731
Fax: 402-472-8466

Topics in
Probability Theory and Stochastic Processes
Steven R. Dunbar

__________________________________________________________________________

Stirling’s Formula derived from the Poisson Distribution

_______________________________________________________________________

Note: These pages are prepared with MathJax. MathJax is an open source JavaScript display engine for mathematics that works in all browsers. See http://mathjax.org for details on supported browsers, accessibility, copy-and-paste, and other features.

_______________________________________________________________________________________________

Rating

Rating

Mathematicians Only: prolonged scenes of intense rigor.

_______________________________________________________________________________________________

Section Starter Question

Section Starter Question

What is the Poisson distribution? What kind of circumstances does a Poisson distribution describe?

_______________________________________________________________________________________________

Key Concepts

Key Concepts

  1. Stirling’s Formula, also called Stirling’s Approximation, is the asymptotic relation
    n! 2πnn+12en.

  2. The formula is useful in estimating large factorial values, but its main mathematical value is in limits involving factorials.
  3. The Poisson distribution with parameter λ is the discrete probability distribution defined on the non-negative integers 0, 1, 2, with probability mass X = k = p(k; λ) = λk k! eλ.

__________________________________________________________________________

Vocabulary

Vocabulary

  1. Stirling’s Formula, also called Stirling’s Approximation, is the asymptotic relation
    n! 2πnn+12en.

  2. The Poisson distribution with parameter λ is the discrete probability distribution defined on the non-negative integers 0, 1, 2, with probability mass X = k = p(k; λ) = λk k! eλ.

__________________________________________________________________________

Mathematical Ideas

Mathematical Ideas

Stirling’s Formula

Stirling’s Formula, also called Stirling’s Approximation, is the asymptotic relation

n! 2πnn+12en.

The formula is useful in estimating large factorial values, but its main mathematical value is in limits involving factorials. Another attractive form of Stirling’s Formula is

n! 2π n n e n.

An improved inequality version of Stirling’s Formula is

2πnn+12en+1(12n+1) < n! < 2πnn+12en+1(12n).

See Stirling’s Formula. in MathWorld.com.

Intuitive Probabilistic Proof

This intuitive argument is adapted from [6, page 171-172] and also from the short article by Hu, [4].

Let X1,X2, be independent Poisson random variables each having mean 1. Let Sn = j=1nX j and then note that the mean and the variance of Sn are equal to n.

Sn = n = n 1 < Sn n = 1n < (Sn n)n 0 1n0 1 2πex22dx 1 2π 1 n

On the other hand Sn is Poisson with mean n, and so

Sn = n = ennn n!

Equating the two expressions for Sn = n, and rearranging obtain

n! 2πnn+12en.

This “proof” relies on having a version of the Central Limit Theorem that has not been proved using Stirling’s Formula! Such a version of the Central Limit Theorem itself is pretty advanced. It also relies on several other advanced facts, such as the distribution of Poisson random variables, the fact that the sum of independent Poisson random variables is again Poisson and the fact that the variance of the sum of independent random variables is the sum of the variances. On the other hand, it is short and simple!

Rigorous Derivation of Stirling’s Formula

The characteristic function with variable 𝜃 of the Poisson distribution p(k; λ) = λk k! eλ with parameter λ is

p̂(𝜃; λ) = k=0p(k; λ)eik𝜃 = eλ(ei𝜃1).

For further properties see Breiman, [1, page 170 ff.], especially Definition 8.26 and Proposition 8.27, or Chung, [2, page 142 ff.], especially item 6 on page 147. The characteristic function is a 2π-periodic function with a series definition which converges uniformly on , meaning we can integrate the series term-by-term on [π,π] to obtain

p(k; λ) = 1 2πππp̂(𝜃,λ)eik𝜃d𝜃 (1)

which is valid for λ > 0 and k = 0, 1, 2, 3,. This is the Fourier inversion formula for the characteristic function. See also Breiman, [1], Theorem 8.39, page 178; Chung, [2], Theorem 6.2.3; or Feller, [3], page 509.

In equation (1) set λ = k, and define Ik by

Ik = kk k! ek = 1 2πππek(ei𝜃1i𝜃)d𝜃k = 0, 1, 2, 3,

Compare this representation to the Gaussian integral with variance k defined as

Jk = 1 2π k = 1 2πek𝜃22d𝜃.

The general plan is to show that Ik Jk, so that then we can assert that

kk k! ek 1 2π k

which can be rearranged to Stirling’s Formula. The detailed plan is to show that this approximation can be expressed as an asymptotic limit.

Break Ik into pieces to make the following definitions

Ik = 1 2π|𝜃|1ek(ei𝜃1i𝜃)d𝜃 + 1 2π1<|𝜃|πek(ei𝜃1i𝜃)d𝜃 = I k(1) + I k(2)

and

Jk = 1 2π|𝜃|1ek𝜃22d𝜃 + 1 2π1<|𝜃|ek𝜃22d𝜃 = J k(1) + J k(2)

Lemma 1. The complex exponential has the following properties and estimates:

  1. For A
    |eA| = eA, (2)
  2. For 𝜃
    |ei𝜃 1 i𝜃| 𝜃22, (3)
  3. For 𝜃
    |ei𝜃 1 i𝜃 + 𝜃22||𝜃|33!, (4)
  4. For A,B
    |eA eB||A B|emax[A,B]. (5)

Proof. Left as exercises. □

Use the triangle inequality for integrals and equation (2) to bound Ik(2) and Jk(2) as

Ik(2) 1 2π1<|𝜃|π|ek(ei𝜃1i𝜃)|d𝜃 1 2π1<|𝜃|πek cos(𝜃)1d𝜃 ek(cos(1)1) 1 2π1<|𝜃|πd𝜃 ek(cos(1)1)

Use a technique similar to the proof of Markov’s inequality to estimate Jk(2) as

Jk(2) = 1 2π|𝜃|>1ek𝜃22d𝜃 1 2π|𝜃|>1|𝜃|ek𝜃22d𝜃 = 1 2π 2 kek2.

Both Ik(2) and Jk(2) tend to zero at an exponential rate.

Now the effort is to estimate the closeness of I1(1) and J1(1). Write

Ik(1) J k(1) = 1 2π11 ek(ei𝜃1i𝜃) ek𝜃22 d𝜃.

For |𝜃| 1, use inequality (4) to derive

| cos(𝜃) 1 + 𝜃22| | cos(𝜃) + i sin(𝜃) 1 i𝜃 + 𝜃22| |ei𝜃 1 i𝜃 + 𝜃22| |𝜃3| 6 𝜃2 6 . (6)

Therefore, cos(𝜃) 1 𝜃23.

Now put these together using inequalities (6) and (4)

|Ik(1) J k(1)| 1 2π11 ek(ei𝜃1i𝜃) ek𝜃22 d𝜃 k11|𝜃|3 3! ek𝜃23d𝜃

Change variables with ϕ = k𝜃 to obtain (the derivation is left as an exercise)

k11|𝜃|3 3! ek𝜃23d𝜃 = 1 6kkk|ϕ3|eϕ23dϕ = 3 2k ek3 2 3ek3 2k

Then putting all these together |Ik Jk| 0. Recalling the definitions of Ik and Jk, this is the same as

k!ek k! 1 2π k 0

as k . This is equivalent to Stirling’s Formula

lim k2π kkkek k! = 1.

An alternate proof using the Lebesgue Dominated Convergence Theorem

Start with the definition of Ik

Ik = kk k! ek = 1 2πππek(ei𝜃1i𝜃)d𝜃.

Make the change of variables y = 𝜃k with dy = d𝜃k to obtain

Ikk = kkk k! ek = 1 2ππkπkek(eiyk1iyk)dy.

Consider the integrand ek(eiyk1iyk). The exponent converges pointwise

k(eiyk 1 iyk) y22

as k by using equation (4), so the integrand converges pointwise to

ek(eiyk1iyk) ey22.

The integrand is bounded pointwise by

|ek(eiyk1iyk)| = ek(cos(yk)1)

using equation (2). Using the half-angle identity, this can be written as

ek(cos(yk)1) = e2k sin 2(y(2k)).

Finally,

e2k sin 2(y(2k)) e2y2π2

if and only if 2k sin 2(y(2k)) 2y2π2 or

sin 2(y(2k)) (yk)2 1 π2

on the domain of integration [πk,πk]. But the function sin 2(y(2k)) (yk)2 has a limit of 14 as y 0, is symmetric around y = 0, and is decreasing on (0,πk). The minimum value of the function occurs at πk and is 1π2. Then using the Lebesgue Dominated Convergence Theorem, [2, page 42] or [1, page 33, Theorem 2.44]

ππek(ei𝜃1i𝜃)d𝜃 ey22dy = 2π.

Putting this together

k!kek k! 2π

as k . This is equivalent to Stirling’s Formula

lim k2π kkkek k! = 1.

Discussion

These proofs establishes the Stirling’s Formula asymptotic limit fairly easily, but are not enough to show the inequality

2πnn+12en+1(12n+1) < n! < 2πnn+12en+1(12n).

In order to establish the inequality requires bounds on the rate of approach of

Ik = 1 2πππek(ei𝜃1i𝜃)d𝜃

to

Jk = 1 2π k = 1 2πek𝜃22d𝜃.

Such an estimate requires bounds on the rate of approach of

ek(eiyk1iyk) ey22

which is possible with careful estimation.

Sources

The heuristic proof using the Central Limit Theorem is adapted from Ross [6, pages 171-172], which in turn is based on Hu [4]. The rigorous proof is adapted from the short article by Pinsky [5].

_______________________________________________________________________________________________

Problems to Work

Problems to Work for Understanding

  1. Show that for A , |eA| = eA
  2. Show that for 𝜃 , |ei𝜃 1 i𝜃| 𝜃22
  3. Show that for 𝜃 , |ei𝜃 1 i𝜃 + 𝜃22||𝜃33!|
  4. Use standard theorems from calculus (either the Fundamental Theorem of Calculus or the Mean Value Theorem) applied to the function f(t) = etA+(1t)B to show that for A,B , |eA eB||A B|emax[A,B]
  5. Show that
    1 6kk k|ϕ3|eϕ23dϕ = 3 2k ek3 2 3ek3 2k

  6. Derive inequalities to estimate the size of the difference
    Ik Jk = 1 2πππek(ei𝜃1i𝜃)d𝜃 1 2π k = 1 2πek𝜃22d𝜃.

    Use these inequalities to derive inequalities for k! refining Stirling’s asymptotic limit formula to an inequality.

  7. In the intuitive proof, the key approximation is Sn = n = n 1 < Sn n = 1n < Zn 0 1n0 1 2πex22dx 1 2π 1 n.

    In the Poisson probability, the interval (for example) (n 23,n + 23] could replace the interval (n 1,n]. But then the value of the integral is proportional to the length 43 of the interval, producing the wrong result. If (for example) (n 12,n + 12] replaces the interval (n 1,n], the result is correct. How does the intuitive proof change when calculating the respective probabilities with an interval of length not equal to 1?

__________________________________________________________________________

Books

Reading Suggestion:

References

[1]   Leo Breiman. Probability. Addison Wesley, 1968.

[2]   Kai Lai Chung. A Course in Probability Theory. Academic Press, 1974.

[3]   William Feller. A Introduction to Probability Theory and It Applications, Volume II, Second Edition, volume II. John Wiley and Sons, second edition, 1971.

[4]   T.-C. Hu. A statistical method of approach to Stirling’s formula. American Statistician, 42:204–205, 1988.

[5]   Mark A. Pinsky. Stirling’s formula via the Poisson distribution. American Mathematical Monthly, 114(3):256–258, March 2007.

[6]   Sheldon M. Ross. Introduction to Probability Models. Elsevier, 6th edition, 1997.

__________________________________________________________________________

Links

Outside Readings and Links:

__________________________________________________________________________

I check all the information on each page for correctness and typographical errors. Nevertheless, some errors may occur and I would be grateful if you would alert me to such errors. I make every reasonable effort to present current and accurate information for public use, however I do not guarantee the accuracy or timeliness of information on this website. Your use of the information from this website is strictly voluntary and at your risk.

I have checked the links to external sites for usefulness. Links to external websites are provided as a convenience. I do not endorse, control, monitor, or guarantee the information contained in any external website. I don’t guarantee that the links are active at all times. Use the links here with the same caution as you would all information on the Internet. This website reflects the thoughts, interests and opinions of its author. They do not explicitly represent official positions or policies of my employer.

Information on this website is subject to change without notice.

Steve Dunbar’s Home Page, http://www.math.unl.edu/~sdunbar1

Email to Steve Dunbar, sdunbar1 at unl dot edu

Last modified: Processed from LATEX source on June 23, 2017