Steven R. Dunbar
Department of Mathematics
203 Avery Hall
University of Nebraska-Lincoln
Lincoln, NE 68588-0130
Probability Theory and Stochastic Processes
Steven R. Dunbar
Local Limit Theorems
Mathematicians Only: prolonged scenes of intense rigor.
Consider a binomial probability value for a large value of the binomial parameter . How would you approximate this probability value? Would you expect this approximation to be uniformly good for all values of ? Explain why or why not?
A local limit theorem describes how the probability mass function of a sum of independent discrete random variables approaches the normal density. We observe that the histogram of a sum of independent Bernoulli random variables resembles the normal density. From the Central Limit Theorem, we see that in standard units the area under the one bar of a binomial histogram may be approximated by the area under a standard normal. Theorems which compare the probability of a discrete random variable in terms of the area under a bar of a histogram to the area under a normal density are often called local limit theorems. This is illustrated in Figure 1
In a way, Pascal laid the foundation for the local limit theorem when he formulated the binomial probability distribution for a Bernoulli random variable with . James Bernoulli generalized the distribution to the case where . De Moivre proved the ﬁrst real local limit theorem for the case in essentially the form of Lemma 9 in The de Moivre-Laplace Central Limit Theorem.. Laplace provided a correct proof for the case with . De Moivre then used the local limit theorem to add up the probabilities that is in an interval of length of order to prove the Central Limit Theorem. See Lemma 10 and following in The de Moivre-Laplace Central Limit Theorem.. Khintchine, Lyanpunov, and Lindeberg proved much more general versions of the Central Limit Theorem using characteristic functions and Fourier transform methods. Historically, the original Local Limit Theorem was overshadowed by the Central Limit Theorem, and forgotten until its revival by Gnedenko in 1948, .
Recall that is a Bernoulli random variable taking on the value or with probability or respectively. Then
is a binomial random variable indicating the number of successes in a composite experiment.
In this section, we study the size of as approaches inﬁnity. We will give an estimate uniform in that is a form of the local limit theorem.
Remark. Note that this is stronger than Lemma 9, the de Moivre-Laplace Binomial Point Mass Limit in the section on the de Moivre-Laplace Central Limit Theorem.. It is stronger in that here the estimate is uniform for all instead of just an interval of order around the mean.
where the last term is uniformly in when . Since
we can simplify this to
where the last asymptotic limit comes from the Moderate Deviations Theorem on .
To see this, note that
Note that the ﬁrst factor goes to by step 2. So consider . This estimate is uniform in for .
uniformly in with .
Finally, factor out and the proof is ﬁnished
Recall that is a sequence of independent random variables which take values with probability and with probability . This is a mathematical model of a fair coin ﬂip game where a results from “heads” on the th coin toss and a results from “tails”. Let and be the number of heads and tails respectively in ﬂips. Then counts the diﬀerence between the number of heads and tails, an excess of heads if positive. The second form of the local limit theorem is useful for estimating the probability that takes a value close to its average .
Remark. The following version of Stirling’s Formula.follows from the statement of the First Form of the Local Limit Theorem. However, note that the proof of the Local Limit Theorem uses the Moderate Deviations Theorem. The proof of the Moderate Deviations Theorem uses the Optimization Extension of the de Moivre-Laplace Central Limit Theorem. Step 1 of the proof of the Optimization Extension uses Stirling’s Formula. So this is not a new proof of Stirling’s Formula for binomial coeﬃcients. It is a long way to derive the asymptotics for binomial coeﬃcients from the usual Stirling Formula.
Proof. Use in the First Form of the Local Limit Theorem. □
Remark. Note that the First Form and the Second Form of the Local Limit Theorem say the same thing in the symmetric case .
Proof. The Second Form of the Local Limit Theorem says that the probability that is in a ﬁxed ﬁnite subset of decreases exponentially as approaches inﬁnity. □
As the Second Form of the Local Limit Theorem indicates, to state a Local Limit Theorem for sums of random variables more general than Bernoulli random variables will take some care. For example, if the summands are all even valued, there is no way the sum can be odd so a Local Limit Theorem will require at least an additional hypothesis while the Central Limit Theorem will still hold.
Deﬁnition. We say that is the maximal span of a density if is the largest integer such that the support of is contained in the aﬃne subset of for some .
Let be the standard normal density.
uniformly in as .
Proof. This was proved by B. V. Gnedenko in 1948, , using characteristic functions. □
McDonald, , has more technical local limit theorems and further references to other generalizations.
The following example shows that any extension of the Local Limit Theorem to nonidentical random variables illustrates is complicated by the same problem of periodicity that already appears in the Second Form of the Local Limit Theorem above.
Example. Suppose the densities are , , . The ﬁrst random variable will be either or , each with probability . For , the random variables will “essentially” diﬀer by . That is, the maximal span is “essentially” , but Gnedenko’s theorem will not hold. In fact, consider the histogram of the convolution of the distributions in Figure 2. This histogram is the distribution of the sum of the ﬁrst of the random variables.
Note that the distribution overall resembles the normal distribution, as expected from the Lindeberg Central Limit Theorem. However closer inspection show that the value of the distribution at symmetric distances around the mean diﬀer. That is the value of the distribution at is greater than the value of the distribution at . This suggests that at least the uniform approach to the normal distribution at integer values will not hold.
This section is adapted from: Heads or Tails, by Emmanuel Lesigne, Student Mathematical Library Volume 28, American Mathematical Society, Providence, 2005, Chapter 9, . The historical remarks and the generalizations of the Local Limit Theorem are adapted from .
The experiment is ﬂipping a coin times, and repeat the experiment times. Then check the probability of a speciﬁc value and compare to the normal probability density. Also compare the logarithmic rate of growth to the predicted rate.
R script for Local Limit Theorem..
Octave script for Local Limit Theorem..
Perl PDL script for Local Limit Theorem..
Scientiﬁc Python script for Local Limit Theorem. .
I check all the information on each page for correctness and typographical errors. Nevertheless, some errors may occur and I would be grateful if you would alert me to such errors. I make every reasonable eﬀort to present current and accurate information for public use, however I do not guarantee the accuracy or timeliness of information on this website. Your use of the information from this website is strictly voluntary and at your risk.
I have checked the links to external sites for usefulness. Links to external websites are provided as a convenience. I do not endorse, control, monitor, or guarantee the information contained in any external website. I don’t guarantee that the links are active at all times. Use the links here with the same caution as you would all information on the Internet. This website reﬂects the thoughts, interests and opinions of its author. They do not explicitly represent oﬃcial positions or policies of my employer.
Information on this website is subject to change without notice.
Steve Dunbar’s Home Page, http://www.math.unl.edu/~sdunbar1
Email to Steve Dunbar, sdunbar1 at unl dot edu
Last modiﬁed: Processed from LATEX source on April 26, 2015