Steven R. Dunbar
Department of Mathematics
203 Avery Hall
Lincoln, NE 68588-0130
http://www.math.unl.edu
Voice: 402-472-3731
Fax: 402-472-8466

Topics in
Probability Theory and Stochastic Processes
Steven R. Dunbar

__________________________________________________________________________

Stirling’s Formula from the Sum of Average Differences

_______________________________________________________________________

Note: To read these pages properly, you will need the latest version of the Mozilla Firefox browser, with the STIX fonts installed. In a few sections, you will also need the latest Java plug-in, and JavaScript must be enabled. If you use a browser other than Firefox, you should be able to access the pages and run the applets. However, mathematical expressions will probably not display correctly. Firefox is currently the only browser that supports all of the open standards.

_______________________________________________________________________________________________ ### Rating

Student: contains scenes of mild algebra or calculus that may require guidance.

_______________________________________________________________________________________________ ### Section Starter Question

How would you estimate $lnk-{\int }_{k-1∕2}^{k+1∕2}lnt\phantom{\rule{0em}{0ex}}\phantom{\rule{0em}{0ex}}dt$ graphically? How would you use Taylor Series to estimate the difference? Compare that estimate with the estimates from the Trapezoidal Approximation of $lnk-{\int }_{k-1∕2}^{k+1∕2}lnt\phantom{\rule{0em}{0ex}}\phantom{\rule{0em}{0ex}}dt$, using the ideas from Stirling’s Formula from Wallis’ Formula and the Trapezoidal Approximation..

_______________________________________________________________________________________________ ### Key Concepts

1. Stirling’s Formula as an asymptotic limit follows from estimation of the difference of $lnk$ and the average of $lnt$ on $\left[k-1∕2,k+1∕2\right]$. Taylor’s Theorem estimates the difference of the mid-value and the average to be $O\left(1∕{k}^{2}\right)$.

__________________________________________________________________________ ### Vocabulary

1. The average of a function $f$ over the interval $\left[a,b\right]$ is
$\frac{1}{b-a}{\int }_{a}^{b}f\left(t\right)\phantom{\rule{0em}{0ex}}dt.$

__________________________________________________________________________ ### Mathematical Ideas

#### Stirling’s Approximation

Theorem 1 (Stirling’s Approximation). For each $n>0$

$n!=\sqrt{2\pi }{n}^{n+1∕2}{e}^{-n}\left(1+{ϵ}_{n}\right).$

where there exists a real constant $A$ so that $|{ϵ}_{n}|<\frac{A}{n}$.

This is equivalent to $n!\sim \sqrt{2\pi }{n}^{n+1∕2}{e}^{-n}$.

Proof. First the proof establishes that there exists a ${c}_{1}\in ℝ$ so that $log\left(n!\right)={c}_{1}+\left(n+1∕2\right)logn-n+O\left(1∕n\right)$.

Expanding the logarithm of $n!$

$\begin{array}{llll}\hfill log\left(n!\right)& =\sum _{k=1}^{n}logk\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\\ \hfill & =\sum _{k=1}^{n}\left(logk-{\int }_{k-1∕2}^{k+1∕2}logt\phantom{\rule{0em}{0ex}}\phantom{\rule{0em}{0ex}}dt+{\int }_{k-1∕2}^{k+1∕2}logt\phantom{\rule{0em}{0ex}}\phantom{\rule{0em}{0ex}}dt\right)\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\\ \hfill & ={\int }_{1∕2}^{n+1∕2}logt\phantom{\rule{0em}{0ex}}\phantom{\rule{0em}{0ex}}dt+\sum _{k=1}^{n}\left(logk-{\int }_{k-1∕2}^{k+1∕2}logt\phantom{\rule{0em}{0ex}}\phantom{\rule{0em}{0ex}}dt\right).\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\end{array}$

Integrating by parts

$\begin{array}{llll}\hfill {\int }_{1∕2}^{n+1∕2}logt\phantom{\rule{0em}{0ex}}\phantom{\rule{0em}{0ex}}dt& ={\left[tlogt-t\right]}_{1∕2}^{n+1∕2}\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\\ \hfill & =\left[\left(n+\frac{1}{2}\right)log\left(n+\frac{1}{2}\right)-\left(n+\frac{1}{2}\right)\right]-\left[\left(\frac{1}{2}\right)log\left(\frac{1}{2}\right)-\left(\frac{1}{2}\right)\right].\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\end{array}$

Rewrite the expression $log\left(n+1∕2\right)$ as

$log\left(n+\frac{1}{2}\right)=logn+log\left(1+\frac{1}{2n}\right)=logn+\frac{1}{2n}+O\left(\frac{1}{{n}^{2}}\right).$

Combining

$\begin{array}{llll}\hfill {\int }_{1∕2}^{n+1∕2}logt\phantom{\rule{0em}{0ex}}dt& =\left(n+\frac{1}{2}\right)logn+\left(n+\frac{1}{2}\right)\frac{1}{2n}-\left(n+\frac{1}{2}\right)+\left[\left(\frac{1}{2}\right)log\left(\frac{1}{2}\right)-\left(\frac{1}{2}\right)\right]+O\left(\frac{1}{n}\right)\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\\ \hfill & ={c}_{1}+\left(n+\frac{1}{2}\right)logn-n+O\left(\frac{1}{n}\right)\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\end{array}$

where the constant ${c}_{1}=-\left[\left(\frac{1}{2}\right)log\left(\frac{1}{2}\right)-\left(\frac{1}{2}\right)\right]$.

Now consider individual terms in the summation:

$\begin{array}{llll}\hfill logk-{\int }_{k-1∕2}^{k+1∕2}logt\phantom{\rule{0em}{0ex}}\phantom{\rule{0em}{0ex}}dt& =logk-\left[{tlogt-t|}_{k-1∕2}^{k+1∕2}\right]\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\\ \hfill & =logk-\left(k+\frac{1}{2}\right)log\left(k+\frac{1}{2}\right)+\left(k-\frac{1}{2}\right)log\left(k-\frac{1}{2}\right)+1\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\\ \hfill & =logk-\frac{1}{2}\left[log\left(k+\frac{1}{2}\right)+log\left(k-\frac{1}{2}\right)\right]+\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\\ \hfill & \phantom{\rule{2em}{0ex}}k\left[log\left(k-\frac{1}{2}\right)-log\left(k+\frac{1}{2}\right)\right]+1\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\\ \hfill & =-\frac{1}{2}\left[log\left(\frac{1}{{k}^{2}}\right)+log\left({k}^{2}-\frac{1}{4}\right)\right]+k\left[log\frac{k\left(1-\frac{1}{2k}\right)}{k\left(1+\frac{1}{2k}\right)}\right]+1\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\\ \hfill & =-\frac{1}{2}log\left(1-4{k}^{-2}\right)-k\left[log\left(1+\frac{1}{2k}\right)-log\left(1-\frac{1}{2k}\right)\right]+1\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\\ \hfill & =O\left(\frac{1}{{k}^{2}}\right)-k\left[\left(\frac{1}{2k}-\frac{1}{8{k}^{2}}+O\left(\frac{1}{{k}^{3}}\right)\right)-\right\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\\ \hfill & \phantom{\rule{2em}{0ex}}\left(\frac{-1}{2k}-\frac{1}{8{k}^{2}}+O\left(\frac{1}{{k}^{3}}\right)\right)]+1\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\\ \hfill & =O\left(\frac{1}{{k}^{2}}\right)-k\left[\frac{1}{k}-\frac{1}{8{k}^{2}}+\frac{1}{8{k}^{2}}+O\left(\frac{1}{{k}^{3}}\right)\right]+1\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\\ \hfill & =O\left(\frac{1}{{k}^{2}}\right).\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\end{array}$

So there exists a value ${c}_{2}$ so that

$\left|logk-{\int }_{k-1∕2}^{k+1∕2}logt\phantom{\rule{0em}{0ex}}\phantom{\rule{0em}{0ex}}dt\right|\le \frac{{c}_{2}}{{k}^{2}}.$

Then set

${c}_{3}=\sum _{k=1}^{\infty }\left(logk-{\int }_{k-1∕2}^{k+1∕2}logt\phantom{\rule{0em}{0ex}}\phantom{\rule{0em}{0ex}}dt\right).$

The tail of the infinite sum is

$\begin{array}{llll}\hfill \sum _{n+1}^{\infty }\left(logk-{\int }_{k-1∕2}^{k+1∕2}logt\phantom{\rule{0em}{0ex}}\phantom{\rule{0em}{0ex}}dt\right)& \le \sum _{n+1}^{\infty }\frac{{c}_{2}}{{k}^{2}}\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\\ \hfill & \le \sum _{n+1}^{\infty }\frac{{c}_{2}}{k\left(k-1\right)}=\frac{{c}_{2}}{n}.\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\end{array}$

The last equation shows that ${\sum }_{1}^{n}\left(logk-\int logt\phantom{\rule{0em}{0ex}}\phantom{\rule{0em}{0ex}}dt\right)={c}_{3}+O\left(\frac{1}{n}\right)$.

Combining all of these equations gives us

$\begin{array}{llll}\hfill log\left(n!\right)& =\left(n+\frac{1}{2}\right)logn-n+{c}_{1}+{c}_{3}+O\left(\frac{1}{n}\right)\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\\ \hfill & =\left(n+\frac{1}{2}\right)logn-n+{c}_{4}+O\left(\frac{1}{n}\right).\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\end{array}$

Note that this is essentially the same form as the conclusion of Lemma 3 in Stirling’s Formula by Euler MacLaurin Summation.. Exponentiating, we have

$n!=\left({n}^{n+1∕2}{e}^{-n}{e}^{{C}_{6}}\right)\left(1+{ϵ}_{n}\right)$

using ${e}^{O\left(\frac{1}{n}\right)}=\left(1+{ϵ}_{n}\right)$.

Recall

Lemma 2 (Wallis’ Formula).

$\underset{n\to \infty }{lim}\frac{{2}^{4n}\cdot {\left(n!\right)}^{4}}{{\left(\left(2n\right)!\right)}^{2}\left(2n+1\right)}=\frac{\pi }{2}$

Proof. See the proofs in Wallis Formula.. □

As before in the proof of Theorem 5 in Stirling’s Formula by Euler MacLaurin Summation. we can conclude that ${e}^{{C}_{6}}=\sqrt{2\pi }$. □

#### Discussion

The Euler-Maclaurin Sum Formula proof of Stirling’s Formula starts with $log\left(n!\right)={\sum }_{j=1}^{n}log\left(j\right)$. The classic proof expresses this as ${\int }_{0}^{n-1}log\left(1+x\right)\phantom{\rule{0em}{0ex}}dx$ with an error term with the Euler-Maclaurin summation formula. The Euler-Maclaurin summation formula is an extension of the Trapezoidal Approximation. (Alternatively, the Euler-Maclaurin summation formula is a result of the Fundamental Theorem of Calculus, summation by parts, and integration by parts.) This allows us to write

 $log\left(n!\right)=nlog\left(n\right)-n+1+\frac{1}{2}log\left(n\right)+{\int }_{1}^{\infty }\frac{{B}_{1}\left(x\right)}{x}\phantom{\rule{0em}{0ex}}\phantom{\rule{0em}{0ex}}dx-{ϵ}_{n}$ (1)

where

${ϵ}_{n}={\int }_{n}^{\infty }\frac{{B}_{1}\left(x\right)}{x}\phantom{\rule{0em}{0ex}}\phantom{\rule{0em}{0ex}}dx$

and ${ϵ}_{n}\to 0$ as $n\to \infty$. Then start from Wallis’ Formula and take logarithms, replacing the logarithms of the factorials with equation (1). This provides an equation for the integral ${\int }_{1}^{\infty }\frac{{B}_{1}\left(x\right)}{x}\phantom{\rule{0em}{0ex}}\phantom{\rule{0em}{0ex}}dx$ which is solved for the value $log\left(\sqrt{2\pi }\right)-1$. Then the equation above can be exponentiated to express Stirling’s Formula.

Here the proof also starts with $log\left(n!\right)={\sum }_{j=1}^{n}log\left(j\right)$. Then the proof expresses this as

${\int }_{1∕2}^{n+1∕2}logt\phantom{\rule{0em}{0ex}}\phantom{\rule{0em}{0ex}}dt+\sum _{k=1}^{n}\left(logk-{\int }_{k-1∕2}^{k+1∕2}logt\phantom{\rule{0em}{0ex}}\phantom{\rule{0em}{0ex}}dt\right).$

Here the error term appears as a sum of terms

$logk-{\int }_{k-1∕2}^{k+1∕2}logt\phantom{\rule{0em}{0ex}}\phantom{\rule{0em}{0ex}}dt$

which is the difference of $logk$ and the average of $logt$ on $\left[k-1∕2,k+1∕2\right]$. Since $logt$ is increasing, it is reasonable to expect the difference should be small. The proof that this difference is $O\left(1∕{k}^{2}\right)$ is established in this section with a Taylor series expansion. Then

$\begin{array}{llll}\hfill log\left(n!\right)& =\left(n+\frac{1}{2}\right)logn-n+{C}_{1}+{C}_{3}+O\left(\frac{1}{n}\right)\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\\ \hfill & =\left(n+\frac{1}{2}\right)logn-n+{C}_{4}+O\left(\frac{1}{n}\right).\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\end{array}$

Together with Wallis’ Formula, this is now enough to establish Stirling’s Formula.

#### Sources

This section is adapted from: Lesigne, pages 36-39, .

_______________________________________________________________________________________________ ### Problems to Work for Understanding

__________________________________________________________________________ ### References

   Emmanuel Lesigne. Heads or Tails: An Introduction to Limit Theorems in Probability, volume 28 of Student Mathematical Library. American Mathematical Society, 2005.

__________________________________________________________________________ __________________________________________________________________________

I check all the information on each page for correctness and typographical errors. Nevertheless, some errors may occur and I would be grateful if you would alert me to such errors. I make every reasonable effort to present current and accurate information for public use, however I do not guarantee the accuracy or timeliness of information on this website. Your use of the information from this website is strictly voluntary and at your risk.

I have checked the links to external sites for usefulness. Links to external websites are provided as a convenience. I do not endorse, control, monitor, or guarantee the information contained in any external website. I don’t guarantee that the links are active at all times. Use the links here with the same caution as you would all information on the Internet. This website reflects the thoughts, interests and opinions of its author. They do not explicitly represent official positions or policies of my employer.

Information on this website is subject to change without notice.