Steven R. Dunbar
Department of Mathematics
203 Avery Hall
University of Nebraska-Lincoln
Lincoln, NE 68588-0130
http://www.math.unl.edu
Voice: 402-472-3731
Fax: 402-472-8466

Topics in
Probability Theory and Stochastic Processes
Steven R. Dunbar

__________________________________________________________________________

Stirling’s Formula Derived from the Gamma Function

_______________________________________________________________________

Note: To read these pages properly, you will need the latest version of the Mozilla Firefox browser, with the STIX fonts installed. In a few sections, you will also need the latest Java plug-in, and JavaScript must be enabled. If you use a browser other than Firefox, you should be able to access the pages and run the applets. However, mathematical expressions will probably not display correctly. Firefox is currently the only browser that supports all of the open standards.

_______________________________________________________________________________________________

Rating

Rating

Mathematicians Only: prolonged scenes of intense rigor.

_______________________________________________________________________________________________

Section Starter Question

Section Starter Question

Do you know of a function defined for, say the positive real numbers, which has the value n! for positive integers n?

_______________________________________________________________________________________________

Key Concepts

Key Concepts

  1. Stirling’s Formula, also called Stirling’s Approximation, is the asymptotic relation
    n! 2πnn+12en.

  2. The formula is useful in estimating large factorial values, but its main mathematical value is in limits involving factorials.
  3. The Gamma function is generally defined as
    Γ(z) =0xz1ex dx.

    For an integer n, Γ(n) = (n 1)! .

__________________________________________________________________________

Vocabulary

Vocabulary

  1. Stirling’s Formula, also called Stirling’s Approximation, is the asymptotic relation
    n! 2πnn+12en.

  2. The Gamma function is generally defined as
    Γ(z) =0xz1ex dx.

    For an integer n, Γ(n) = (n 1)! .

__________________________________________________________________________

Mathematical Ideas

Mathematical Ideas

Stirling’s Formula, also called Stirling’s Approximation, is the asymptotic relation

n! 2πnn+12en.

The formula is useful in estimating large factorial values, but its main mathematical value is in limits involving factorials. Another attractive form of Stirling’s Formula is:

n! 2π n n e n.

An improved inequality version of Stirling’s Formula is

2πnn+12en+1(12n+1) < n! < 2πnn+12en+1(12n). (1)

See Stirling’s Formula. in MathWorld.com.

First we formally define the Gamma function and its most important property. Then we give a heuristic argument for Stirling’s formula for the Gamma function using asymptotics of integrals, based on some notes by P. Garrett, [1]. Here we rigorously derive Stirling’s Formula using the Gamma function and estimates of the logarithm function, based on the short note by R. Michel, [2].

Definition of the Gamma Function

Lemma 1 (Gamma Function).

n! =0xnex dx.

Proof. The proof is by induction. Start with n = 0,

0x0ex dx =0ex dx = 1 = 0!.

Then in general, integrating by parts

0xnex dx = n0xn1ex dx.

Inductively,

n! =0xnex dx.

Remark. The Gamma function is generally defined as

Γ(z) =0xz1ex dx.

Then the same integration by parts shows Γ(z) = (z 1)Γ(z 1) . For an integer n, Γ(n) = (n 1)! .

Heuristic Derivation of Stirling’s Formula from Asymptotics of Integrals

Observe that

Γ(z + 1) =0xzex dx =0ex+z log x dx.

Note that x + z log x has a unique critical point at x = z. The critical point is a maximum since the second derivative has value 1z < 0 there. The idea is to replace x + z log x by its second degree Taylor polynomial based at z, then evaluate the resulting integral. Note that the second degree Taylor polynomial is the quadratic polynomial best approximating a function. The idea is that most of the contribution to the integral comes from the maximum, and because of the negative exponential, contribution away from the maximum is slight.

So expand x + z log x in a second-degree Taylor polynomial at the critical point z:

x + z log x z + z log z 1 2!z(x z)2.

Thus we can approximate the Gamma function with

Γ(z + 1) = zΓ(z) =0ex+z log x dx  (2) 0ez+z log z 1 2!z(xz)2 dx = ezzz0e(xz)22z dx ezzze(xz)22z dx.

Note that last evaluation of the integral is over the whole real line. By Lemma 4 of DeMoivre-Laplace Central Limit Theorem.

0e(xz)22z dx 1 2zez4.

This is very small for large z, so heuristically the asymptotics of the integral over the whole line is of the same order as the integral over [0,).

To simplify the integral, replace x by zu. Notice that a factor of z cancels from both sides

Γ(z) = ezzzez(u1)22 du.

Make another change of variables x = z(u 1) so that

ezzzez(u1)22 du = ezzz1 zex2 2 dx = ezzz2π z .

The last equality is derived in Evaluation of the Gaussian Density Integral..

Simplifying,

Γ(s) 2πezzz12.

As an alternative, one can apply Laplace’s method (for asymptotics of integrals) to the integral in (2) and derive the same conclusion somewhat more rigorously.

Rigorous Derivation of Stirling’s Formula

Lemma 2.

n! = nnneng n(y) dy

where

gn(y) = 1 + y nneyn1 (n,)(y).

Proof. In Lemma 1 make the substitution x = ny + n (or equivalently y = x n n) with dx = ndy to give

0xnex dx =n(yn + n)ne(yn+n)ndy = nnnenn(yn + 1)ne(yn) dy = nnneng n(y) dy .

Remark. This means that what remains is to show the integral g n(y) dy approaches the asymptotic constant 2π .

Lemma 3. For |x| < 1,

log(1 + x) x + 1 2x2 1 3 |x|3 1 |x|.


PIC

Figure 1: A graph of the functions in Lemma 3 with | log(1 + x) x + x2 2 | in red and 1 3 |x|3 1|x| in blue.

Remark. Note further that

|x|3 1 |x| |x|2 (3)

for |x| 12. This will be used later to simplify the expression on the right side of the inequality in Lemma 3.

Proof. Expanding log(1 + x) in a Taylor series and applying the triangle inequality

log(1 + x) x + 1 2x2 k=3|x|3 k .

Grossly overestimating each term by using the least denominator and summing as a geometric series

k=3|x|3 k 1 3 |x|3 1 |x|.

Lemma 4.

|ea eb|eb |a b|e|ab|.

Proof.

|ea eb| = eb|eab 1|

Notice that ex 1 = k=1xk k! = x k=1xk1 k! x k=1 xk1 (k1)! = x k=0xk k! = xex. Alternatively let f(x) = ex 1 and g(x) = xex . Then f(0) = 0 = g(0), and f(x) = ex ex + xex = g(x) for x > 0 . Likewise f(x) = ex ex + xex = g(x) for x < 0 . Therefore ex 1 = f(x) g(x) = xex for all x . Then

eb|eab 1|eb |a b|e|ab|.

Lemma 5. For |y|n 2

|gn(y) ey22||y|3 ney26.

Proof. For |y|n 2

|gn(y) ey22| = 1 + y nneyn1 (n,)(y) ey22 = elog1+ y nnyn 1(n,)(y) ey22 .  Using |y|n 2 = elog1+ y nnyn ey22 .  Using Lemma 4 ey22 log 1 + y nn yn + y22 elog1+ y nnyn+y22 ,  then with rules of logarithms = ey22 n log 1 + y n y n + y2 2n enlog1+ y n y n+y2 2n ,  and using Lemma 3 ey22 n 1 3 |y|3n32 1 |y|nen1 3|y|3n32 1|y|n .  Making a coarse estimate on the fraction ey22 n 1 3 |y|3n32 1 (n2)nen1 3 |y|3n32 1|y|n = ey22 2 3 |y|3 n12en1 3 |y|3n32 1|y|n .  Use the remark after Lemma 3 = ey22 2 3 |y|3 n12e1 3|y|2   Combine the exponents and coarsely over-estimate the fraction |y|3 ney26.

Lemma 6.

lim ngn(y) = ey22

where the limit is uniform on compact subsets of .

Proof. For |y|n 2 use Lemma5

|gn(y) ey22||y|3 ney26.

Note that |y|3ey26 has a maximum value of 27e32 .

Let K be a compact subset of with M be a bound so that K {x : |x| < M} . Let ϵ > 0 be given. Let n be so large that M < n2 and 27e32n < ϵ . Then for all y K, |gn(y) ey22||y|3 n ey26 27e32n < ϵ . □

Lemma 7. For x > 1

log(1 + x) x 5 6 x2 2 + x.


PIC

Figure 2: A graph of the functions in Lemma 7 with log(1 + x) in red and x (56)(x2)(2 + x) in blue.

Proof. Consider f(x) = x 5 6 x2 2+x log(1 + x), with f(x) = x x2x+4 6(1+x)(2+x)2 . The only critical point on the domain x > 1 is x = 0, and f(x) < 0 for 1 < x < 0, f(x) > 0 for x > 0 . Then f(x) has a global minimum value of 0 at x = 0 . Hence log(x) x 5 6 x2 2+x . □

Remark. Unfortunately, although the proof is straightforward the origin of the rational function on the right is unmotivated.

Lemma 8.

0 gn(y) e|y|6,|y| > 1 2n

Proof. For y > n,

log(gn(y)) = n log(1 + y n) yn 5 6 y2 2 + yn

using Lemma 7. For |y| > 1 2n and on the domain y > n, 1 < 2 + yn 2 + |y|n, so

5|y| 2 + yn 5|y| 2 + |y|n.

Now since |y| > 1 2n, it is true that 4|y| > 2n, so 5|y| > 2n + |y| or 5|y| 2+|y|n > n . Finally multiplying through by |y|6,

5 6 y2 2 + yn 5 6 y2 2 + |y|n < |y|n 6 < |y| 6 .

Theorem 9 (Stirling’s Formula).

n! 2πnn+12en.

Proof. By Lemma 2

n! = nnneng n(y) dy

where

gn(y) = 1 + y nneyn1 (n,)(y).

From Lemma 6

lim ngn(y) = ey22

uniformly on compact sets and gn(y) is integrable by Lemma 8. Hence by the Lebesgue Convergence Theorem

lim ng n(y) dy =ey22 dy

and since ey22 dy = 2π it follows that

lim n n! 2πnnnen = lim ng n(y) dy 2π = 1.

Asymptotic Expansions

Lemma 10. For x > 0

0 < 1 + 1 12 x2 1 + x 1 x + 1 2 log(1 + x) x4 120.

Remark. Unfortunately, although the proof is straightforward the origin of the function on the left is unmotivated.


PIC

Figure 3: A graph of the functions in Lemma 10 with 1 + 1 12 x2 1+x 1 x + 1 2 log(1 + x) in red and x4 120 in blue.

Proof. Let f(x) = 2x + 1 6 x3 1+x (x + 2) log(1 + x) for x > 1 so f(x) = 1 6 x3 (1+x)2 1 2 x2 21+x log(1 + x) x+2 x+1 + 2 . Then f(0) = 0, and f(0) = 0 . Further f(x) = 1 3 x3 (1+x)3 and f(x) > 0 for x > 0 . Therefore, f(x) > 0 for x > 0 and dividing through by 2x yields the left inequality.

For the right side, consider h(x) = x5 60 f(x) . Again h(0) = h(0) = 0 . Further h(x) = x3 3 1 3 x3 (1+x)3 > 0 for x > 1 . The right side inequality follows immediately. □

Lemma 11. For x > 0,

ex 1 + x + 1 2x2 + 1 6x3ex.

Proof. For x > 0, expanding ex in a Taylor series

ex = 1 + x + 1 2x2 + k=3xk k! = 1 + x + 1 2x2 + 1 6x3 k=3xk3 k!6 = 1 + x + 1 2x2 + 1 6x3 k=0 xk (k + 3)!6 1 + x + 1 2x2 + 1 6x3 k=0xk k!

since (k + 3)!6 k! . □

Lemma 12.

1 6 e124 123 1 9440

Remark. Numerically 1 6 e124 123 0.0001005542925 and 1 9440 0.0001059322034, so the difference is about 5 × 106 .

Proof. I don’t have a rigorous proof based on, say, elementary comparisons in the rational number system. In fact, it is not clear how the denominator 9440 arises. However, as will be seen in the proof of Theorem 13 what is really needed is that 1 6 e124 123 is a finite constant. Finding a rational number with unit numerator and 4-digit denominator to bound the constant is simply a convenience. □

Theorem 13. For n 2,

n! 2πnn+12en 1 1 12n 1 288n2 + 1 9940n3.

Proof. Let an = n! 2πnn+12en . Then an an+1 = n+1 n n+12e1 and log an an+1 = (n + 12) log(1 + 1n) 1 . Using the left inequality in Lemma 10 with x = 1n

log an an+1 1 12n(n + 1).

Then

log an an+r = k=nn+r1 log ak ak+1 1 12 k=nn+r1 1 k(k + 1) = 1 12 1 n 1 n + r.

Now let r , noting that lim ran+r = 1, so log(an) 1 12n and an e1(12n) . Now use Lemma 11 and Lemma 12 to yield

an 1 + 1 12n + 1 2 1 (12n)2 + 1 6 1 (12n)3e1(12n) 1 + 1 12n + 1 2 1 (12n)2 + 1 6 1 (12n)3e1(122) 1 + 1 12n + 1 2 1 (12n)2 + 1 9440n3.

Then

n! 2πnn+12en 1 1 12n 1 2 1 (12n)2 + 1 9440n3.

Likewise, using the right inequality in Lemma 10 with x = 1n

log an an+1 1 12(n(n + 1)) 1 120 1 n4.

Then

log an an+r = k=nn+r1 log ak ak+1 1 12 k=nn+r1 1 k(k + 1) 1 120 k=nn+r1 1 k4 = 1 12 1 n 1 n + r 1 120 k=nn+r1 1 k4.

Again let r , noting that lim ran+r = 1 to obtain

log(an) 1 12n 1 120 k=n 1 k4 = 1 12n 1 120 1 n4 k=n+1 1 k4 1 12n 1 120 1 n4 1 360n3

because k=n+1 1 k4 n 1 x4 dx using right-box Riemann sums with width 1.

Let rn = 1 12n 1 120 1 n4 1 360n3 . Hence an ern . By a well-known estimate ern 1 + r n . Therefore,

an 1 + 1 12n 1 120 1 n4 1 360n3

Equivalently,

n! 2πnn+12en 1 1 12n 1 360n3 1 120 1 n4.

Since

1 360n3 + 1 120 1 n4 1 2 1 (12n)2 + 1 9440n3

the two inequalities can be summarized as

n! 2πnn+12en 1 1 12n 1 2 1 (12n)2 + 1 9440n3.

Remark. The proof actually shows the slightly stronger bounds

1 360n3 1 120 1 n4 n! 2πnn+12en 1 1 12n 1 2 1 (12n)2 + 1 9440n3.

Remark. Using similar reasoning, from the well-known estimate ern 1 + r n + 1 2rn2, one can derive the estimate

n! 2πnn+12en 1 1 12n 1 288n2 1 360n3 + 1 108n4

valid for n 3 .

Remark. Note that for n = 1,

n! 2πnn+12en = e 2π 13 12 = 1 + 1 12

since e 2π 1.0844 and 13 12 1.08334 . Similarly for n = 2

n! 2πnn+12en = e2 2π232 25 24 = 1 + 1 12 2

since e2 2π232 1.042207 and 25 24 1.041664 . From the remark above, for n 3

n! 2πnn+12en 1 1 12n 1 288n2 1 360n3 1 108n4 > 0.

Then for all n 1,

n! 2πnn+12en 1 + 1 12n

which is a better bound than e1(2n+1) quoted in equation (1). In order to see this bound, consider f(x) = log(1 + x) 2x 2+x, defined for x > 1. Then f(x) = x2 (1+x)(2+x)2 and f(0) = 0, f(x) > 0 for x > 0. Then log(1 + 1 12n) > 2 24n+1 > 1 12n+1. Hence 1 + 1 12n > e1(12n+1) .

Alternate Derivation of Stirling’s Asymptotic Formula

Note that xtex has its maximum value at x = t. That is, most of the value of the Gamma Function comes from values of x near t. Therefore use a partition of the Gamma Function with t > 0, f(x) = xtex for x > 0, and A = {x : |x t| t2}. Then

Γ(t + 1) =0xtex dx =t23t2f(x) dx +01 A(x)f(x) dx

where 1A() is the indicator function (or characteristic function) of A. For x A, 1 4(x t)2t2, so we have 1A(x) 4(x t)2t2. Then

1 1 Γ(t + 1)t23t2xtex dx 1 Γ(t + 1){x:|xt|t2}4(x t)2 t2 xtex dx 4 Γ(t + 1)t20(x t)2xtex dx

Expanding (x t)2 and using Γ(z + 1) = zΓ(z) from the remark following Lemma1 yields

0(xt)2xtex dx = (t + 2)(t + 1)Γ(t + 1) 2t(t + 1)Γ(t + 1) + t2Γ(t + 1).

Then simplifying the right side

1 1 Γ(t + 1)t23t2xtex dx 42 + t t2

Making the change of variables x = yt + t and setting gt(y) = (1 + yt)teyt for y > t just as in the proof of Lemma2, we get

lim t ttt Γ(t + 1)ett2t2g t(y) dy = 1. (4)

Now using |x| 12 and Lemma3

log(1 + x) x + 1 2x2 1 3 |x|3 1 |x|2 3|x|3.

Then using Lemma5

gt(y) ey22 |y|3 t ey26

for |y|t2. Then

t2t2g t(y) dy ey22 dy 1 tt2t2|y|3ey26 dy +|y|>t2ey2 2 dy = 3tet24 2 36et24 t + 36 t +|y|>t2ey2 2 dy

Therefore,

lim tt2t2g t(y) dy =ey22 dy = 2π.

Combining this with the limit in equation 4, we have

lim tΓ(t + 1)et 2πtt+12 = 1.

Sources

The main part of this section is adapted from the short note by R. Michel, [2]. The alternate derivation is adapted from the even shorter note by R. Michel, [3].

_______________________________________________________________________________________________

Problems to Work

Problems to Work for Understanding

  1. Show that ey22 dy = 2π .

__________________________________________________________________________

Books

Reading Suggestion:

References

[1]   Paul Garrett. Asymptotics of integrals. Online, accessed November 2012, July 2011.

[2]   Reinhard Michel. On Stirling’s formula. American Mathematical Monthly, 109(4):388–390, April 2002.

[3]   Reinhard Michel. The (n + 1)th proof of Stirling’s Formula. American Mathematical Monthly, 115(9):844–845, November 2008.

__________________________________________________________________________

Links

Outside Readings and Links:

__________________________________________________________________________

I check all the information on each page for correctness and typographical errors. Nevertheless, some errors may occur and I would be grateful if you would alert me to such errors. I make every reasonable effort to present current and accurate information for public use, however I do not guarantee the accuracy or timeliness of information on this website. Your use of the information from this website is strictly voluntary and at your risk.

I have checked the links to external sites for usefulness. Links to external websites are provided as a convenience. I do not endorse, control, monitor, or guarantee the information contained in any external website. I don’t guarantee that the links are active at all times. Use the links here with the same caution as you would all information on the Internet. This website reflects the thoughts, interests and opinions of its author. They do not explicitly represent official positions or policies of my employer.

Information on this website is subject to change without notice.

Steve Dunbar’s Home Page, http://www.math.unl.edu/~sdunbar1

Email to Steve Dunbar, sdunbar1 at unl dot edu

Last modified: Processed from LATEX source on November 26, 2012