Steven R. Dunbar
Department of Mathematics
203 Avery Hall
Lincoln, NE 68588-0130
http://www.math.unl.edu
Voice: 402-472-3731
Fax: 402-472-8466

Topics in
Probability Theory and Stochastic Processes
Steven R. Dunbar

__________________________________________________________________________

The Moderate Deviations Result

_______________________________________________________________________

Note: To read these pages properly, you will need the latest version of the Mozilla Firefox browser, with the STIX fonts installed. In a few sections, you will also need the latest Java plug-in, and JavaScript must be enabled. If you use a browser other than Firefox, you should be able to access the pages and run the applets. However, mathematical expressions will probably not display correctly. Firefox is currently the only browser that supports all of the open standards.

_______________________________________________________________________________________________ ### Rating

Mathematicians Only: prolonged scenes of intense rigor.

_______________________________________________________________________________________________ ### Section Starter Question

__________________________________________________________________________ ### Key Concepts

1. For any sequence ${a}_{n}$ with $\sqrt{n}\ll {a}_{n}\ll n$ we have
$ℙn\left[{S}_{n}-pn\ge {a}_{n}\right]\to 0$

but neither the Central Limit Theorem nor the Large Deviations Principle tells us how fast the convergence is, nor what the precise rate of growth is for ${a}_{n}$. Making this precise is the domain of Moderate Deviations Theorem.

2. Precisely, if ${a}_{n}\to \infty$, and $\underset{n\to \infty }{lim}\frac{{a}_{n}}{{n}^{1∕6}}=0$ then
$ℙn\left[\frac{{S}_{n}}{n}-p\ge \sqrt{p\left(1-p\right)}\frac{{a}_{n}}{\sqrt{n}}\right]\sim \frac{1}{{a}_{n}\sqrt{2\pi }}{e}^{-{a}_{n}^{2}∕2}.$

__________________________________________________________________________ ### Vocabulary

1. Moderate deviations results are refinements of the Central Limit Theorem
$ℙn\left[\frac{{S}_{n}}{n}-p\ge \sqrt{p\left(1-p\right)}\frac{{a}_{n}}{\sqrt{n}}\right]\sim \frac{1}{{a}_{n}\sqrt{2\pi }}{e}^{-{a}_{n}^{2}∕2}.$

when ${a}_{n}=o\left({n}^{1∕6}\right)$.

__________________________________________________________________________ ### Mathematical Ideas

Recall that ${X}_{k}$ is a Bernoulli random variable taking on the value $1$ or $0$ with probability $p$ or $1-p$ respectively. Then

${S}_{n}=\sum _{k=1}^{n}{X}_{i}$

is a binomial random variable indicating the number of successes in a composite experiment.

The Large Deviations Estimate shows that the probability of large deviation events of the type

$ℙn\left[\frac{{S}_{n}}{n}-p>x\right]$

i.e. the sample mean exceeds the mean by more than $x$, decays exponentially in $x$. Equivalently, the probability $ℙn\left[{S}_{n}-np\ge nx\right]$ that the partial sum ${S}_{n}$ exceeds its mean by more than $nx$ is exponentially small in $x$. The de Moivre-Laplace Central Limit Theorem tells us the probability that the partial sum exceeds its average by an order of $\sqrt{n}$. Precisely

$ℙn\left[\frac{{S}_{n}}{n}-p\ge \frac{x}{\sqrt{n}}\right]\to 1-\Phi \left(x\right)>0.$

Equivalently, the probability $ℙn\left[{S}_{n}-np\right]\ge \sqrt{n}x$ for partial sums approaches the standard normal distribution. This implies that for any sequence ${a}_{n}$ with $\sqrt{n}\ll {a}_{n}\ll n$ we still have

$ℙn\left[{S}_{n}-pn\ge {a}_{n}\right]\to 0$

and neither the Central Limit Theorem nor the Large Deviations Estimate tells us how fast the convergence is, nor what the precise rate of growth is for ${a}_{n}$. Making this precise is the domain of Moderate Deviations Theorem.

The Moderate Deviations Theorem is due to Harald Cramér in 1938.

First we have to prove two supplementary results, each of which is interesting in its own right.

Proposition 1 (“Optimization” extension of de Moivre-Laplace Central Limit Theorem). Assume

1. For $0\le k\le n$, define ${\delta }_{n}\left(k\right)$ by
$\left(\genfrac{}{}{0.0pt}{}{n}{k}\right){p}^{k}{\left(1-p\right)}^{k}=\frac{1}{\sqrt{2\pi p\left(1-p\right)n}}{e}^{\left(\frac{-{\left(k-np\right)}^{2}}{2np\left(1-p\right)}\right)}\left(1+{\delta }_{n}\left(k\right)\right).$

2. Let ${c}_{n}$ be a positive real sequence with $\underset{n\to \infty }{lim}{c}_{n}=0$.
3. Let ${I}_{n}^{\prime }=\left\{k\in ℤ:|k-np|<{c}_{n}{n}^{2∕3}\right\}$.

Then

$\underset{n\to \infty }{lim}\underset{k\in {I}_{n}^{\prime }}{max}|{\delta }_{n}\left(k\right)|=0.$ Figure 1: Comparison of the binomial distribution with $n=12$, $p=4∕10$ with the normal distribution with mean $np$ and variance $np\left(1-p\right)$.

Remark. In Figure 1 the amount ${\delta }_{n}\left(k\right)$ is the small relative error between the height of the normal distribution curve and the height of the binomial distribution histogram over the integer $k$.

Remark. Compare the statement of this proposition to the statement of the de Moivre-Laplace Binomial Point Mass Limit, Lemma 9 in de Moivre Laplace Central Limit Theorem.. Here the domain of the maximum is ${I}_{n}^{\prime }=\left\{k\in ℤ:|k-np|<{c}_{n}{n}^{2∕3}\right\}$ which is slightly larger than the domain in the de Moivre-Laplace Binomial Point Mass Limit, ${I}_{n}=\left\{k\in ℤ:|k-np|.

Proof.

1. Recall from the de Moivre-Laplace Theorem (see step 2 of the proof of Lemma 9 in de Moivre Laplace Central Limit Theorem.) that from Stirling’s Formula $\begin{array}{c}\left(\genfrac{}{}{0.0pt}{}{n}{k}\right){p}^{k}{\left(1-p\right)}^{n-k}=\frac{n!}{k!\left(n-k\right)!}{p}^{k}{\left(1-p\right)}^{n-k}\\ =\frac{1}{\sqrt{2\pi }}\sqrt{\frac{n}{k\left(n-k\right)}}{\left(\frac{np}{k}\right)}^{k}{\left(\frac{n\left(1-p\right)}{\left(n-k\right)}\right)}^{n-k}\left(\frac{1+{ϵ}_{n}}{\left(1+{ϵ}_{k}\right)\left(1+{ϵ}_{n-k}\right)}\right).\end{array}$

where ${ϵ}_{n}, ${ϵ}_{k}, ${ϵ}_{n=k} for some constant $A$.

2. For $k\in {I}_{n}^{\prime }$ $\begin{array}{c}\frac{n}{\left(np+{c}_{n}{n}^{2∕3}\right)\left(n\left(1-p\right)+{c}_{n}{n}^{2∕3}\right)}\le \\ \frac{n}{k\left(n-k\right)}\le \\ \frac{n}{\left(np-{c}_{n}{n}^{2∕3}\right)\left(n\left(1-p\right)-{c}_{n}{n}^{2∕3}\right)},\end{array}$

$\begin{array}{c}\frac{1}{n}\cdot \frac{1}{\left(p+{c}_{n}{n}^{-1∕3}\right)\left(\left(1-p\right)+{c}_{n}{n}^{-1∕3}\right)}\le \\ \frac{n}{k\left(n-k\right)}\le \\ \frac{1}{n}\cdot \frac{1}{\left(p-{c}_{n}{n}^{-1∕3}\right)\left(\left(1-p\right)-{c}_{n}{n}^{-1∕3}\right)},\end{array}$

$\begin{array}{c}\frac{1}{np\left(1-p\right)}\cdot \frac{1}{\left(1+\frac{{c}_{n}{n}^{-1∕3}}{p}\right)\left(1+\frac{{c}_{n}{n}^{-1∕3}}{1-p}\right)}\le \\ \frac{n}{k\left(n-k\right)}\le \\ \frac{1}{np\left(1-p\right)}\cdot \frac{1}{\left(1-\frac{{c}_{n}{n}^{-1∕3}}{p}\right)\left(1-\frac{{c}_{n}{n}^{-1∕3}}{1-p}\right)}.\end{array}$

Compare this to step 3 of the proof of Lemma 9 in de Moivre Laplace Central Limit Theorem..

3. Therefore, for $k\in {I}_{n}^{\prime }$

This follows from the One-Term Geometric Series Expansion and the Square-Root Expansion Proposition in the section Big-Oh Algebra.

Compare this to steps 4,5 of the proof of Lemma 9 in de Moivre Laplace Central Limit Theorem..

4. Since $k\in {I}_{n}^{\prime }$, and . Compare this to step 7 of the proof of Lemma 9 in de Moivre Laplace Central Limit Theorem..
5. Using the Taylor series expansion for the logarithm

Compare this to step 8 of the proof of Lemma 9 in de Moivre Laplace Central Limit Theorem..

6. Thus
 (2)

See the Exponential Expansion Proposition in the section Big-Oh Algebra.

7. Step 10 of the proof of Lemma 9 in de Moivre Laplace Central Limit Theorem. showed why
 (3)
8. Now combining equations (1), (2) and (3) above into step 1, we get

where ${c}_{n}^{\prime }=max\left({c}_{n}{n}^{-1∕3},{c}_{n}^{3},{n}^{-1}\right)$.

Proposition 2. Assume

1. ${k}_{n}$ and ${\ell }_{n}$ are two sequences with ${k}_{n}<{\ell }_{n}$ for all $n$.
2. ${k}_{n}=np+o\left({n}^{2∕3}\right)$ and ${\ell }_{n}=np+o\left({n}^{2∕3}\right)$; i.e., ${k}_{n}=np+{c}_{n}^{\prime }{n}^{2∕3}$ where ${c}_{n}^{\prime }\to 0$ as $n\to \infty$ and ${\ell }_{n}=np+{c}_{n}^{\prime \prime }{n}^{2∕3}$ where ${c}_{n}^{\prime \prime }\to 0$ as $n\to \infty$.
3. Let ${a}_{n}=\frac{{k}_{n}-np}{\sqrt{np\left(1-p\right)}}$ and ${b}_{n}=\frac{{\ell }_{n}-np}{\sqrt{np\left(1-p\right)}}$.

Then

$ℙn\left[{k}_{n}\le {S}_{n}\le {\ell }_{n}\right]\sim \frac{1}{\sqrt{2\pi }}{\int }_{{a}_{n}}^{{b}_{n}}{e}^{-{x}^{2}∕2}\phantom{\rule{0em}{0ex}}dx,$

as $n\to \infty$.

Remark. If $\left({a}_{n}\right)$ and $\left({b}_{n}\right)$ converge respectively to $a$ and $b$ such that $a then this proposition becomes the de Moivre-Laplace Central Limit Theorem.

Proof.

1. Take $n$ so large that $0\le {k}_{n}<{\ell }_{n}\le n$.
2. Let $h\left(n\right)=\frac{1}{\sqrt{np\left(1-p\right)}}$. Then by the Optimization Proposition 1
$ℙn\left[{S}_{n}=j\right]=\frac{h\left(n\right)}{\sqrt{2\pi }}exp\left(\frac{-{\left(j-np\right)}^{2}}{2np\left(1-p\right)}\right)\left(1+{\delta }_{n}\left(j\right)\right).$

and

 $ℙn\left[{k}_{n}\le {S}_{n}<{\ell }_{n}\right]=\frac{h\left(n\right)}{\sqrt{2\pi }}\sum _{j={k}_{n}}^{{\ell }_{n-1}}exp\left(\frac{-{\left(j-np\right)}^{2}}{2np\left(1-p\right)}\right)\left(1+{\delta }_{n}\left(j\right)\right).$ (4)

The hypotheses on the sequences $\left({k}_{n}\right)$ and $\left({\ell }_{n}\right)$ along with Proposition 1 imply that the sequence $\left({\delta }_{n}\left(j\right)\right)$ converges uniformly to zero when ${k}_{n}\le j\le {\ell }_{n}$.

3. Therefore it suffices to show that
$h\left(n\right)\sum _{j={k}_{n}}^{{\ell }_{n}-1}exp\left(\frac{-{\left(j-np\right)}^{2}}{2np\left(1-p\right)}\right)\sim {\int }_{{a}_{n}}^{{b}_{n}}{e}^{-{x}^{2}∕2}\phantom{\rule{0em}{0ex}}dx.$

4. Set
$x\left(j\right)=\frac{j-np}{\sqrt{np\left(1-p\right)}}.$

Then ${a}_{n}=x\left({k}_{n}\right)$ and ${b}_{n}=x\left({\ell }_{n}\right)$.

5. The claim is:
 $h\left(n\right)\sum _{j={k}_{n}}^{{\ell }_{n}-1}exp\left(\frac{-{\left(j-np\right)}^{2}}{2np\left(1-p\right)}\right)-{\int }_{{a}_{n}}^{{b}_{n}}{e}^{-{x}^{2}∕2}\phantom{\rule{0em}{0ex}}dx=o\left({\int }_{{a}_{n}}^{{b}_{n}}{e}^{-{x}^{2}∕2}\phantom{\rule{0em}{0ex}}dx\right).$ (5)

This claim will follow by considering the Riemann sums for the integral of ${e}^{-{x}^{2}∕2}$.

6. In the case $\left({a}_{n}\right)>0$, $\begin{array}{c}h\left(n\right)exp\left(-\frac{x{\left(j+1\right)}^{2}}{2}\right)<{\int }_{x\left(j\right)}^{x\left(j+1\right)}exp\left(-{x}^{2}∕2\right)\phantom{\rule{0em}{0ex}}dx\\

For ${k}_{n}\le j\le {\ell }_{n}$ obtain

$\begin{array}{cc}& 0\le h\left(n\right)\sum _{j={k}_{n}}^{{\ell }_{n-1}}exp\left(\frac{-x{\left(j\right)}^{2}}{2}\right)-{\int }_{{a}_{n}}^{{b}_{n}}{exp}^{-{x}^{2}∕2}\phantom{\rule{0em}{0ex}}dx\\ & \le h\left(n\right)\left(exp\left(-{a}_{n}^{2}∕2\right)-exp\left(-{b}_{n}^{2}∕2\right)\right)& \text{(6)}\end{array}$

7. Also $\begin{array}{cc}& {\int }_{{a}_{n}}^{{b}_{n}}{exp}^{-{x}^{2}∕2}\phantom{\rule{0em}{0ex}}dx\ge \left(\frac{1}{{b}_{n}}\right){\int }_{{a}_{n}}^{{b}_{n}}x{exp}^{-{x}^{2}∕2}\phantom{\rule{0em}{0ex}}dx\\ & =\left(\frac{1}{{b}_{n}}\right)\left(exp\left(-{a}_{n}^{2}∕2\right)-exp\left(-{b}_{n}^{2}∕2\right)\right).& \text{(7)}\end{array}$

8. Note $h\left(n\right)=o\left({b}_{n}^{-1}\right)$ since ${b}_{n}=o\left({n}^{1∕6}\right)$. Then combining (6) and (7) yields (5)

Theorem 3 (Moderate Deviations Theorem). Suppose

1. $\left({a}_{n}\right)$ is a sequence of real numbers,
2. ${a}_{n}\to \infty$ as $n\to \infty$ and
3. $\underset{n\to \infty }{lim}\frac{{a}_{n}}{{n}^{1∕6}}=0$.

Then

$ℙn\left[\frac{{S}_{n}}{n}-p\ge \sqrt{p\left(1-p\right)}\frac{{a}_{n}}{\sqrt{n}}\right]\sim \frac{1}{{a}_{n}\sqrt{2\pi }}{e}^{-{a}_{n}^{2}∕2}.$

Remark. Step 6 of the proof of the Moderate Deviations Theorem shows that

$\frac{1}{\sqrt{2\pi }}{\int }_{{a}_{n}}^{{b}_{n}}{e}^{-{x}^{2}∕2}\phantom{\rule{0em}{0ex}}dx\sim \frac{1}{{a}_{n}\sqrt{2\pi }}{e}^{-{a}_{n}^{2}∕2}.$

so that an equivalent result is that

$ℙn\left[\frac{{S}_{n}}{n}-p\ge \sqrt{p\left(1-p\right)}\frac{{a}_{n}}{\sqrt{n}}\right]\sim \frac{1}{{a}_{n}\sqrt{2\pi }}{e}^{-{a}_{n}^{2}∕2}.$

Remark. The de Moivre-Laplace Central Limit Theorem tells us that as $n\to \infty$

$ℙn\left[\frac{{S}_{n}}{n}-p\ge \sqrt{p\left(1-p\right)}\frac{a}{\sqrt{n}}\right]\sim \Phi \left(a\right)=\frac{1}{\sqrt{2\pi }}{\int }_{a}^{\infty }{e}^{-\frac{{x}^{2}}{2}}\phantom{\rule{0em}{0ex}}dx$

The moderate deviations result tells us that this estimate remains true when $a$ is allowed to approach $\infty$ at a slow enough rate.

Proof.

1. The hypothesis 3 says that ${a}_{n}\to \infty$ less quickly than ${n}^{1∕6}$.
2. Let ${d}_{n}=\sqrt{{a}_{n}}$. Then $\underset{n\to \infty }{lim}\frac{{d}_{n}}{{a}_{n}}=0$ and so ${d}_{n}=o\left({a}_{n}\right)$.
3. Let ${k}_{n}=⌈np+\sqrt{np\left(1-p\right)}{a}_{n}⌉$ and ${\ell }_{n}=⌈np+\sqrt{np\left(1-p\right)}\left({a}_{n}+{d}_{n}\right)⌉$. A schematic diagram of where all the sequences sit relative to each other is below: 4. Event $\left[{S}_{n}\ge {k}_{n}\right]=\left[\frac{{S}_{n}}{n}-p\ge \frac{{k}_{n}}{n}-p\right]=\left[\frac{{S}_{n}}{n}-p\ge \frac{\sqrt{p\left(1-p\right)}}{\sqrt{n}}{a}_{n}\right]$. Thus, $\begin{array}{llll}\hfill ℙn\left[\frac{{S}_{n}}{n}-p\ge \frac{\sqrt{p\left(1-p\right)}}{\sqrt{n}}{a}_{n}\right]& =ℙn\left[{S}_{n}\ge {k}_{n}\right]\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\\ \hfill & =ℙn\left[{k}_{n}\le {S}_{n}<{\ell }_{n}\right]+ℙn\left[{S}_{n}\ge {\ell }_{n}\right].\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\end{array}$

Step 5 below will take care of the first summand. Step 7 below will take care of the second summand.

5. By hypothesis 3, ${a}_{n}=o\left({n}^{1∕6}\right)$ and so ${k}_{n},{\ell }_{n}=np+o\left({n}^{2∕3}\right)$. From Proposition 3, set
${a}_{n}^{\prime }=\frac{{k}_{n}-np}{\sqrt{np\left(1-p\right)}},{b}_{n}=\frac{{\ell }_{n}-np}{\sqrt{np\left(1-p\right)}},$

and so

$ℙn\left[{k}_{n}\le {S}_{n}<{\ell }_{n}\right]\sim \frac{1}{\sqrt{2\pi }}{\int }_{{a}_{n}^{\prime }}^{{b}_{n}}{e}^{-{x}^{2}∕2}\phantom{\rule{0em}{0ex}}dx.$

This allows us to say that

$ℙn\left[{k}_{n}\le {S}_{n}<{\ell }_{n}\right]\sim \frac{1}{\sqrt{2\pi }}{\int }_{{a}_{n}}^{{b}_{n}}{e}^{-{x}^{2}∕2}\phantom{\rule{0em}{0ex}}dx-\frac{1}{\sqrt{2\pi }}{\int }_{{a}_{n}}^{{a}_{n}^{\prime }}{e}^{-{x}^{2}∕2}\phantom{\rule{0em}{0ex}}dx.$

Step 6 below will take care of the first summand. Step 7 below will take care of the second summand.

6. The claim is that
${\int }_{{a}_{n}}^{{b}_{n}}{e}^{-{x}^{2}∕2}\phantom{\rule{0em}{0ex}}dx\sim \frac{1}{{a}_{n}}{e}^{-{a}_{n}^{2}∕2}.$

1. Notice that
${\int }_{{a}_{n}}^{{b}_{n}}{e}^{-{x}^{2}∕2}\phantom{\rule{0em}{0ex}}dx\le \frac{1}{{a}_{n}}{\int }_{{a}_{n}}^{\infty }x{e}^{-{x}^{2}∕2}\phantom{\rule{0em}{0ex}}dx=\frac{1}{{a}_{n}}{e}^{-{a}_{n}^{2}∕2}.$

2. We also have that ${b}_{n}\ge {a}_{n}+{d}_{n}$, since normalizing the ceiling is at least as big as normalizing the argument of the ceiling. Thus, $\begin{array}{llll}\hfill {\int }_{{a}_{n}}^{{b}_{n}}{e}^{-{x}^{2}∕2}\phantom{\rule{0em}{0ex}}dx& \ge {\int }_{{a}_{n}}^{{a}_{n}+{d}_{n}}{e}^{-{x}^{2}∕2}\phantom{\rule{0em}{0ex}}dx\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\\ \hfill & \ge \frac{1}{{a}_{n}+{d}_{n}}{\int }_{{a}_{n}}^{{a}_{n}+{d}_{n}}x{e}^{-{x}^{2}∕2}\phantom{\rule{0em}{0ex}}dx\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\\ \hfill & =\frac{1}{{a}_{n}+{d}_{n}}\left(exp\left(\frac{-{a}_{n}^{2}}{2}\right)-exp\left(\frac{-{\left({a}_{n}+{d}_{n}\right)}^{2}}{2}\right)\right).\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\end{array}$

Divide by $\frac{1}{{a}_{n}}exp\left(\frac{-{a}_{n}^{2}}{2}\right)$ to get on the right hand side:

$\begin{array}{llll}\hfill & =\frac{{a}_{n}}{{a}_{n}+{d}_{n}}-\frac{{a}_{n}}{{a}_{n}+{d}_{n}}\frac{exp\left(\frac{-{\left({a}_{n}+{d}_{n}\right)}^{2}}{2}\right)}{exp\left(\frac{-{a}_{n}^{2}}{2}\right)}\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\\ \hfill & \to 1-0=1.\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\end{array}$

Now combine steps 6a and6b to get the claim of step 6.

7. The claim is that
${\int }_{{a}_{n}}^{{a}_{n}^{\prime }}{e}^{-{x}^{2}∕2}\phantom{\rule{0em}{0ex}}dx=o\left(\frac{1}{{a}_{n}}{e}^{-{a}_{n}^{2}∕2}\right).$

The fact that $0\le {a}_{n}^{\prime }-{a}_{n}\le {\left(np\left(1-p\right)\right)}^{-1∕2}$ directly implies that

${\int }_{{a}_{n}}^{{a}_{n}^{\prime }}{e}^{-{x}^{2}∕2}\phantom{\rule{0em}{0ex}}dx\le \frac{1}{\sqrt{np\left(1-p\right)}}exp\left(\frac{-{a}_{n}^{2}}{2}\right)$

by approximating the integral with a 1-box left or lower Riemann sum. Divide through by $\frac{1}{{a}_{n}}exp\left(\frac{-{a}_{n}^{2}}{2}\right)$.

$\frac{{\int }_{{a}_{n}}^{{a}_{n}^{\prime }}{e}^{-{x}^{2}∕2}\phantom{\rule{0em}{0ex}}dx}{\frac{1}{{a}_{n}}exp\left(\frac{-{a}_{n}^{2}}{2}\right)}<\frac{{a}_{n}}{\sqrt{np\left(1-p\right)}}\to 0,$

since ${a}_{n}=o\left({n}^{1∕6}\right)$.

8. The claim is that
$ℙn\left[{S}_{n}\ge {\ell }_{n}\right]=o\left(\frac{1}{{a}_{n}}{e}^{-{a}_{n}^{2}∕2}\right).$

1. By the Large Deviations Theorem we have
$ℙn\left[{S}_{n}\ge {\ell }_{n}\right]\le exp\left(-n{h}_{+}\left(\sqrt{p\left(1-p\right)}\frac{{b}_{n}}{\sqrt{n}}\right)\right),$

where for $ϵ\to 0$. Thus,

since ${b}_{n}=o\left({n}^{1∕2}\right)$.

2. Notice that $\begin{array}{llll}\hfill exp\left(\frac{-{b}_{n}^{2}}{2}\right)& \le exp\left(\frac{-{\left({a}_{n}+{d}_{n}\right)}^{2}}{2}\right)\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\\ \hfill & =o\left(exp\left(\frac{-{d}_{n}^{2}}{2}\right)exp\left(\frac{-{a}_{n}^{2}}{2}\right)\right).\phantom{\rule{2em}{0ex}}& \hfill & \phantom{\rule{2em}{0ex}}\end{array}$
3. We can see that $exp\left(\frac{-{d}_{n}^{2}}{2}\right)\le \frac{1}{{a}_{n}}$ by our choice of ${d}_{n}$. (Note that ${d}_{n}>\sqrt{2ln{a}_{n}}$.)

This concludes step 8.

Example. Take ${a}_{n}={n}^{1∕8}$, so that ${a}_{n}\to \infty$ and $\underset{n\to \infty }{lim}\frac{{a}_{n}}{{n}^{1∕6}}=\underset{n\to \infty }{lim}{n}^{-1∕24}=0$. Take $p=1∕2$. Take $n=1{0}^{4}$, so ${a}_{1{0}^{4}}=\sqrt{10}$ and

$ℙn\left[\frac{{S}_{1{0}^{4}}}{1{0}^{4}}-\frac{1}{2}\ge \frac{1}{2}\frac{\sqrt{10}}{\sqrt{1{0}^{4}}}\right]=ℙn\left[{S}_{1{0}^{4}}\ge 5000+50\sqrt{10}\right].$

 R 1-pbinom(5000+50*sqrt(10)-1, 10^4,0.5) 0.0008156979 (1/(sqrt(10*2*pi)))*exp(-(sqrt(10))^2/2 0.0008500367 Octave 1-binocdf(5000+50*sqrt(10)-1, 10^4, 0.5) 8.1570e-04 (1/(sqrt(10*2*pi))*exp(-sqrt(10)^2/2) 8.5004e-04

Using the deMoivre-Laplace Central Limit Theorem in R:  1- pnorm(5000+50*sqrt (10), mean=5000, sd=50)  gives $0.0007827011$.

#### Sources

The explanatory remarks at the beginning comparing the Moderate Deviations Theorem to the Large Deviations Theorem and the Central Limit Theorem are from the survey article by Mörters. This section is adapted from: Heads or Tails, by Emmanuel Lesigne, Student Mathematical Library Volume 28, American Mathematical Society, Providence, 2005, Chapter 8. . .

_______________________________________________________________________________________________ ### Algorithms, Scripts, Simulations

#### Algorithm

The experiment is flipping a coin $n$ times, and repeat the experiment $k$ times. Then check the probability of moderate deviations.

#### Scripts

R
p <$-$ 0.5
n <$-$ 10000
k <$-$ 1000
coinFlips <$-$ array( 0+(runif(nk) <= p), dim=c(n,k))
# 0+ coerces Boolean to numeric
headsTotal <$-$ colSums(coinFlips)   # 0..n binomial rv sample, size k

an <$-$ nˆ(1/8)
mu <$-$ p
stddev <$-$ sqrt(p(1$-$p)n)
moddev <$-$ mu + stddev(an)
prob <$-$ sum( 0+(headsTotal > moddev) )/
theoretical <$-$ ( 1/(sqrt(2pi)an) )exp$-$(an)ˆ2/2 )
cat(sprintf(”Empirical_probability:_%f_n”, prob ))
cat(sprintf(”Moderate_Deviations_Theorem_estimate:_%f_n”, theoretical))
Octave
p = 0.5;
n = 10000;
k = 1000;

coinFlips = rand(n,k) <= p;
headsTotal = sum(coinFlips);  # 0..n binomial rv sample, size k

an = nˆ(1/8);
mu = pn;
stddev = sqrt(p(1$-$p)n);
moddev = mu + stddevan;
prob = sum( headsTotal > moddev)/k;
theoretical = ( 1/(sqrt(2pi)an) )exp$-$(an)ˆ2/2 );
disp(”Empirical_probability:”), disp( prob )
disp(”Moderate_Deviations_Theorem_estimate:”), disp( theoretical )
Perl
use PDL::NiceSlice;
use PDL::Constants qw(PI);

\$p = 0.5;
\$n = 10000;
\$k = 1000;

\$coinFlips = random( \$k, \$n ) <= \$p;    #note order of dims!!
\$coinFlips$-$>transpose$-$>sumover;     # 0..n binomial r.v. sample, size k

#note transpose, PDL likes x (row) direction for implicitly threaded operations

\$an     = \$n∗∗( 1 / 8 );
\$mu     = \$p  \$n;
\$stddev = sqrt( \$p  ( 1 $-$ \$p )  \$n );
\$moddev = \$mu + \$stddev  \$an;

\$prob = ( ( \$headsTotal > \$moddev )$-$>sumover ) / \$k;
\$theoretical = ( 1 / ( sqrt( 2  PI )  \$an ) )  exp$-$( \$an∗∗2 ) / 2 );

print ”Empirical_probability:_”,               \$prob,        ”n”;
print ”Moderate_Deviations_Theorem_estimate:”, \$theoretical, ”n”;
SciPy
import scipy

p = 0.5
n = 10000
k = 1000

coinFlips = scipy.random.random((n,k))<= p
# Note Booleans True for Heads and False for Tails
headsTotal = scipy.sum(coinFlips, axis = 0) # 0..n binomial r.v. sample, size k
# Note how Booleans act as 0 (False) and 1 (True)

an = n∗∗(1./8.)
mu = p  n
stddev = scipy.sqrt( p  ( 1$-$p ) n )
moddev = mu + stddev  an

prob = (scipy.sum( headsTotal  > moddev)).astype(’float’)/k
# Note the casting of integer type to float to get float
theoretical = ( 1/(scipy.sqrt(2scipy.pi)an))scipy.exp($-$(an∗∗2)/2)

print ”Empirical_probability:_”, prob
print ”Moderate_Deviations_Theorem_estimate:”, theoretical

__________________________________________________________________________ ### Problems to Work for Understanding

1. Using the Proposition 1 show that if $\left({a}_{n}\right)$ and $\left({b}_{n}\right)$ converge respectively to $a$ and $b$ such that $a then this proposition becomes the de Moivre-Laplace Central Limit Theorem.
2. Explain why

if ${b}_{n}=o\left({n}^{1∕2}\right)$.

3. Explain why for $k\in {I}_{n}^{\prime }$, and .
4. Explain why
$\frac{exp\left(\frac{-{\left({a}_{n}+{d}_{n}\right)}^{2}}{2}\right)}{exp\left(\frac{-{a}_{n}^{2}}{2}\right)}\to 0.$

5. Explain why
$exp\left(\frac{-{\left({a}_{n}+{d}_{n}\right)}^{2}}{2}\right)=o\left(exp\left(\frac{-{d}_{n}^{2}}{2}\right)exp\left(\frac{-{a}_{n}^{2}}{2}\right)\right).$

__________________________________________________________________________ ### References

   Emmanuel Lesigne. Heads or Tails: An Introduction to Limit Theorems in Probability, volume 28 of Student Mathematical Library. American Mathematical Society, 2005.

   Peter Mörters. Large deivation theory and applications. http://people.bath.ac.uk/maspm/LDP.pdf, November 2008. Cramér’s theorem, large deviations, moderate deviations.

__________________________________________________________________________ __________________________________________________________________________

I check all the information on each page for correctness and typographical errors. Nevertheless, some errors may occur and I would be grateful if you would alert me to such errors. I make every reasonable effort to present current and accurate information for public use, however I do not guarantee the accuracy or timeliness of information on this website. Your use of the information from this website is strictly voluntary and at your risk.

I have checked the links to external sites for usefulness. Links to external websites are provided as a convenience. I do not endorse, control, monitor, or guarantee the information contained in any external website. I don’t guarantee that the links are active at all times. Use the links here with the same caution as you would all information on the Internet. This website reflects the thoughts, interests and opinions of its author. They do not explicitly represent official positions or policies of my employer.

Information on this website is subject to change without notice.