Steven R. Dunbar
Department of Mathematics
203 Avery Hall
University of Nebraska-Lincoln
Lincoln, NE 68588-0130
http://www.math.unl.edu
Voice: 402-472-3731
Fax: 402-472-8466

Math 489/Math 889
Stochastic Processes and
Advanced Mathematical Finance
Dunbar, Fall 2010

__________________________________________________________________________

The Absolute Excess of Heads over Tails

_______________________________________________________________________

Rating

Mathematically Mature: may contain mathematics beyond calculus with proofs.

_______________________________________________________________________________________________

QuestionofDay

Question of the Day

What does the law of averages have to say about the probability of having a fixed lead of say 20 Heads or more over Tails or 20 Tails or more over Heads at the end of a coin flipping game of some fixed duration? What does the Weak Law of Large Numbers have to say about having a fixed lead? What does the Weak Law have to say about having a proportional lead, say 1%? What does the Central Limit Theorem have to say about the lead?

_______________________________________________________________________________________________

Key Concepts

The probability that the number of heads exceeds the number of tails or the number of tails exceeds the number of heads in a sequence of coin-flips by some fixed amount can be estimated with the Central Limit Theorem and the probability gets close to $1$ as the number of tosses grows large.
The probability that the number of heads exceeds the number of tails or the number of tails exceeds the number of heads in a sequence of coin-flips by some fixed proportion can be estimated with the Central Limit Theorem and the probability gets close to $0$ as the number of tosses grows large.

__________________________________________________________________________

Vocabulary

The half-integer correction, also called the continuity correction arises because the distribution of the binomial distribution is a discrete distribution, while the standard normal distribution is a continuous distribution.

__________________________________________________________________________

Mathematical Ideas

Introduction

Probability theory generally has two classes of theorems about the results of coin-tossing games and therefore random walks:

Those theorems that tell how well-behaved and natural are the outcomes of typical coin-tossing games and random walks. The Weak Law of Large Numbers, the Strong Law of Large Numbers and the Central Limit Theorem fall into this category.
Those theorems that tell how strange and unnatural are the outcomes of typical coin-tossing games and random walks. The Arcsine Law and the Law of the Iterated Logarithm are good examples in this category.

In this section we will ask two related questions about the net fortune in a coin-tossing game:

What is the probability of an excess of a fixed number of heads over tails or tails over heads at some fixed time in the coin-flipping game?
What is the probability that the number of heads exceeds the number of tails or the number of tails exceeds the number of heads by some fixed fraction of the number of tosses?

Using the Central Limit Theorem, we will be able to provide precise answers to each question and then to apply the ideas to interesting questions in gambling and finance.

The Half-Integer Correction to the Central Limit Theorem

Often when using the Central Limit Theorem to approximate a discrete distribution, especially the binomial distribution, we adopt the half-integer correction, also called the continuity correction. The correction arises because the binomial distribution has a discrete distribution while the standard normal distribution has a continuous distribution. For any integer $s$ and real value $h$ with $0 \leq h < 1$ the binomial random variable $S_{n}$ has $ℙ [| S_{n} | \leq s] = ℙ [| S_{n} | \leq s + h]$ , yet the corresponding Central Limit Theorem approximation with the standard normal cumulative distribution function, $ℙ [| Z | \leq (s + h) ∕ \sqrt{n}]$ increases as $h$ increases from $0$ to $1$ . It is customary to take $h = 1 ∕ 2$ to interpolate the difference. This choice is also justified by looking at the approximation of the binomial with the normal.

Symbolically, the half-integer correction to the Central Limit Theorem is

\begin{array}{l} ℙ [a \leq S_{n} \leq b] & \approx \int_{((a - 1 ∕ 2) - n p) ∕ \sqrt{n p q}}^{((b + 1 ∕ 2) - n p) ∕ \sqrt{n p q}} \frac{1}{\sqrt{2 π}} exp (- u^{2} ∕ 2) d u \\ = ℙ [((a - 1 ∕ 2) - n p) ∕ \sqrt{n p q} \leq Z \leq ((b + 1 ∕ 2) - n p) ∕ \sqrt{n p q}] \end{array}

for integers $a$ and $b$ .

Figure 1:

The half-integer correction

The absolute excess of heads over tails

Consider the sequence of independent random variables $Y_{i}$ which take values $1$ with probability $1 ∕ 2$ and $- 1$ with probability $1 ∕ 2$ . This is a mathematical model of a fair coin flip game where a $1$ results from “heads” on the $i$ th coin toss and a $- 1$ results from “tails”. Let $H_{n}$ and $L_{n}$ be the number of heads and tails respectively in $n$ flips. Then $T_{n} = \sum_{i = 1}^{n} Y_{i} = H_{n} - L_{n}$ counts the difference between the number of heads and tails, an excess of heads if positive, and a “negative excess”, i.e. a deficit, if negative. Rather than the clumsy extended phrase “the number of heads exceeds the number of tails or the number of tails exceeds the number of heads” we can say “the absolute excess of heads $| T_{n} |$ .” The value $T_{n}$ also represents the net “winnings”, positive or negative, of a gambler in a fair coin flip game.

Corollary 1. Under the assumption that $Y_{i} = + 1$ with probability $1 ∕ 2$ and $Y_{i} = - 1$ with probability $1 ∕ 2$ , and $T_{n} = \sum_{i = 1}^{n} Y_{i}$ , then for an integer $s$

ℙ [| T_{n} | > s] \approx ℙ [| Z | \geq (s + 1 ∕ 2) ∕ \sqrt{n}]

where $Z$ is a standard normal random variable with mean $0$ and variance $1$ .

Proof. Note that $μ = E [Y_{i}] = 0$ and $σ^{2} = Var [Y_{i}] = 1$ .

\begin{array}{l} ℙ [| T_{n} | > s] & = 1 - ℙ [- s \leq T_{n} \leq s] \\ = 1 - ℙ [- s - 1 ∕ 2 \leq T_{n} \leq s + 1 ∕ 2] \\ = 1 - ℙ [(- s - 1 ∕ 2) ∕ \sqrt{n} \leq T_{n} ∕ \sqrt{n} \leq (s + 1 ∕ 2) ∕ \sqrt{n}] \\ \approx 1 - ℙ [(- s - 1 ∕ 2) ∕ \sqrt{n} \leq Z \leq (s + 1 ∕ 2) ∕ \sqrt{n}] \\ = ℙ [| Z | \geq (s + 1 ∕ 2) ∕ \sqrt{n}] \end{array}

□

The crucial step occurs at the approximation, and uses the Central Limit Theorem. More precise statements of the Central Limit Theorem such as the Berry-Esseen inequality can turn the approximation into a inequality.

If we take $s$ to be fixed we now have the answer to our first question: The probability of an absolute excess of heads over tails greater than a fixed amount in a fair game of duration $n$ approaches $1$ as $n$ increases.

The Central Limit Theorem in the form of the half-integer correction above provides an alternative proof of the Weak Law of Large Numbers for the specific case of the binomial random variable $T_{n}$ . In fact,

\begin{array}{l} ℙ [|\frac{T_{n}}{n}| > ϵ] & \approx ℙ [| Z | \geq (ϵ n + 1 ∕ 2) ∕ \sqrt{n}] \\ = ℙ [| Z | \geq ϵ \sqrt{n} + (1 ∕ 2) ∕ \sqrt{n}] \to 0 \end{array}

as $n \to \infty$ .

Rewriting $ℙ [|\frac{T_{n}}{n}| > ϵ] = ℙ [|T_{n}| > ϵ n]$ this restatement of the Weak Law actually provides the answer to our second question: The probability that the absolute excess of heads over tails is greater than a fixed fraction of the flips in a fair game of duration $n$ approaches $0$ as $n$ increases.

Finally, this gives an estimate on the central probability in a binomial distribution.

Corollary 2.

ℙ [T_{n} = 0] \approx ℙ [| Z | < (1 ∕ 2) ∕ \sqrt{n}] \to 0

as $n \to \infty$ .

We can estimate this further

\begin{array}{l} ℙ [| Z | < (1 ∕ 2) ∕ \sqrt{n}] & = \frac{1}{\sqrt{2 π}} \int_{- 1 ∕ (2 \sqrt{n})}^{1 ∕ (2 \sqrt{n})} e^{- u^{2} ∕ 2} d u \\ = \frac{1}{\sqrt{2 π}} \int_{- 1 ∕ (2 \sqrt{n})}^{1 ∕ (2 \sqrt{n})} 1 - u^{2} ∕ 2 + u^{4} ∕ 8 + \dots d u \\ = \frac{1}{\sqrt{2 π}} \frac{1}{\sqrt{n}} - \frac{1}{24 \sqrt{2 π}} \frac{1}{n^{3 ∕ 2}} + \frac{1}{640 \sqrt{2 π}} \frac{1}{n^{5 ∕ 2}} + \dots . \end{array}

So we see that $ℙ [T_{n} = 0]$ goes to zero at the rate of $1 ∕ \sqrt{n}$ .

Illustration 1

What is the probability that the number of heads exceeds the number of tails by more than $20$ or the number of tails exceeds the number of heads by more than $20$ after $500$ tosses of a fair coin? By the proposition, this is:

ℙ [| T_{n} | > 20] \approx ℙ [| Z | \geq 20.5 ∕ \sqrt{500}] = 0.3477 .

This is a reasonably large probability, and is larger than many people would expect.

Here is a graph of the probability of at least $s$ excess heads in $500$ tosses of a fair coin:

Figure 2:

Probability of

s

excess heads in

500

tosses

Illustration 2

What is the probability that there is “about the same number of heads as tails” in 500 tosses? Here we interpret “about the same” as within $5$ , that is, an absolute difference of 1% or less of the number of tosses. Note that since $500$ is even, so the difference in the number of heads and tails cannot be an odd number, so must be either $0$ , $2$ or $4$ .

ℙ [| S_{500} | < 5] \approx ℙ [| Z | \leq 5.5 ∕ \sqrt{500}] = 0.1943

so it would be somewhat unusual (in that it occurs in less than 20% of games) to have the number of heads and tails so close.

Illustration 3

Suppose you closely follow a stock recommendation source whose methods are based on technical analysis. You accept every bit of advice from this source about trading stocks. You choose $10$ stocks to buy, sell or hold every day based on the recommendations. Each day for each stock you will gain or lose money based on the advice. Note that it is possible to gain money even if the advice says the stocks will decrease in value, say by short-selling or using put options. How good can this strategy be? We will make this vague question precise by asking “How good does the information from the technical analysis have to be so that the probability of losing money over a year’s time is $1$ in $10, 000$ ?”

The $10$ stocks over $260$ business days in a year means that there $2, 600$ daily gains or losses. Denote each daily gain or loss as $X_{i}$ , if the advice is correct you will gain $X_{i} > 0$ and if the advice is wrong you will lose $X_{i} < 0$ . We want the total change $X_{annual} = \sum_{i = 1}^{2600} X_{i} > 0$ and we will measure that by asking that $ℙ [X_{annual} < 0]$ be small. In the terms of this section, we are interested in the complementary probability of an excess of successes over failures.

We assume that the changes are random variables, identically distributed, independent and the moments of all the random variables are finite. We will make specific assumptions about the distribution later, for now these assumptions are sufficient to apply the Central Limit Theorem. Then the total change $X_{annual} = \sum_{i = 1}^{2600} X_{i}$ is approximately normally distributed with mean $μ = 2600 \cdot E [X_{1}]$ and variance $σ^{2} = 2600 \cdot Var [X_{1}]$ . Note that here again we are using the uncentered and unscaled version of the Central Limit Theorem. In symbols

ℙ [a \leq \sum_{i = 1}^{2600} X_{i} \leq b] \approx \frac{1}{\sqrt{2} π σ^{2}} \int_{a}^{b} exp (- \frac{{(u - μ)}^{2}}{2 σ^{2}}) d u .

We are interested in

ℙ [\sum_{i = 1}^{2600} X_{i} \leq 0] \approx \frac{1}{\sqrt{2} π σ^{2}} \int_{- \infty}^{0} exp (- \frac{{(u - μ)}^{2}}{2 σ^{2}}) d u .

By the change of variables $v = (u - μ) ∕ σ$ , we can rewrite the probability as

ℙ [X_{annual} \leq 0] = \frac{1}{\sqrt{2} π} \int_{- \infty}^{- μ ∕ σ} exp (- v^{2} ∕ 2) d v = Φ (- \frac{μ}{σ})

so that the probability depends only on the ratio $- μ ∕ σ$ . We desire that $Φ (- μ ∕ σ) = ℙ [X_{annual} < 0] 1 ∕ 10, 000$ . Then we can solve for $- μ ∕ σ \approx - 3.7$ . Since $μ = 2600 \cdot E [X_{1}]$ and $σ^{2} = 2600 \cdot Var [X_{1}]$ , we calculate that for the total annual change to be a loss we must have $E [X_{1}] \approx (3.7 ∕ \sqrt{2600}) \cdot \sqrt{Var [X_{1}]} = 0.07 \cdot \sqrt{Var [X_{1}]}$ .

Now we consider what the requirement $E [X_{1}] = 0.07 \cdot \sqrt{Var [X_{1}]}$ means for specific distributions. If we assume that the individual changes $X_{i}$ are normally distributed with a positive mean, then we can use $E [X_{1}] ∕ \sqrt{Var [X_{1}]} = 0.07$ to calculate that $ℙ [X_{1}] < 0 \approx 0.47210$ , or about $47 %$ . Alternatively, if we assume that the individual changes $X_{i}$ are binomial random variables with $ℙ [X_{1} = 1] = p$ , then $E [X_{1}] = 2 p - 1$ and $Var [X_{1}] = 4 p (1 - p)$ . We can use $2 p - 1 = E [X_{1}] = 0.07 Var [X_{1}] = 0.07 \cdot (4 p (1 - p))$ to solve for $p$ . The result is $p = 0.53491$ .

In either case, this means that any given piece of advice only has to have a $53 %$ chance of being correct in order to have a perpetual money-making machine. Compare this with the strategy of using a coin flip to provide the advice. Since we don’t observe any perpetual money-making machines, we conclude that any advice about stock picking must be less than $53 %$ reliable or about the same as flipping a coin.

Now suppose that instead we have a computer algorithm predicting stock movements for all publicly traded stocks, of which there are about $2, 000$ . Suppose further that we wish to restrict the chance that $ℙ [X_{annual}] < 1 0^{- 6}$ , that is 1 chance in a million. Then we can repeat the analysis to show that the computer algorithm would only need to have $ℙ [X_{1}] < 0 \approx 0.49737$ , practically indistinguishable from a coin flip, in order to make money. This provides a statistical argument against the utility of technical analysis for stock price prediction. Money-making is not sufficient evidence to distinguish ability in stock-picking from coin-flipping.

Sources

This section is adapted from the article “Tossing a Fair Coin” by Leonard Lipkin. The discussion of the continuity correction is adapted from Partial Sums and the Central Limit Theorem. in the Virtual Laboratories in Probability and Statistics. The third example in this section is adapted from a presentation by Jonathan Kaplan of D.E. Shaw and Co. in summer 2010.

_______________________________________________________________________________________________

Problems to Work for Understanding

1. What is the approximate probability that the number of heads is within $10$ of the number of tails, that is, a difference of 2% or less of the number of tosses in 500 tosses?
2. What is the approximate probability that the number of heads is within $20$ of the number of tails, that is, a difference of 4% or less of the number of tosses in 500 tosses?
3. What is the approximate probability that the number of heads is within $25$ of the number of tails, that is, a difference of 5% or less of the number of tosses in 500 tosses?
4. Derive and then graph a simple power function that gives the approximate probability that the number of heads is within $x$ of the number of tails in 500 tosses, for $0 \leq x \leq 500$ .
1. What is the probability that the number of heads is within $10$ of the number of tails, that is, a difference of 1% or less of the number of tosses in 1000 tosses?
2. What is the probability that the number of heads is within $10$ of the number of tails, that is, a difference of 0.5% or less of the number of tosses in 2000 tosses?
3. What is the probability that the number of heads is within $10$ of the number of tails, that is, a difference of 0.2% or less of the number of tosses in 5000 tosses?
4. Derive and then graph a simple power function that gives the approximate probability that the number of heads is within $x$ of the number of tails in 500 tosses, for $0 \leq x \leq 500$ .
Derive the rate, as a function of $n$ , that the probability of heads exceeds tails by a fixed value $s$ approaches $1$ as $n \to \infty$ .
Derive the rate, as a function of $n$ , that the probability of heads exceeds tails by a fixed fraction $ϵ$ approaches $0$ as $n \to \infty$ .

__________________________________________________________________________

Reading Suggestion:

References

[1] William Feller. An Introduction to Probability Theory and Its Applications, Volume I, Third Edition, volume I. John Wiley and Sons, third edition edition, 1973. QA 273 F3712.

[2] Emmanuel Lesigne. Heads or Tails: An Introduction to Limit Theorems in Probability, volume 28 of Student Mathematical Library. American Mathematical Society, 2005.

[3] Leonard Lipkin. Tossing a fair coin. The College Mathematics Journal, 34(2):128–133, March 2003.

__________________________________________________________________________

Outside Readings and Links:

Virtual Laboratories in Probability and Statistics.. Search the page for Random Walk Simulation and run the Last Visit to Zero experiment.
Virtual Laboratories in Probability and Statistics.. Search the page for Ballot Experiment and run the Ballot Experiment.

__________________________________________________________________________

I check all the information on each page for correctness and typographical errors. Nevertheless, some errors may occur and I would be grateful if you would alert me to such errors. I make every reasonable effort to present current and accurate information for public use, however I do not guarantee the accuracy or timeliness of information on this website. Your use of the information from this website is strictly voluntary and at your risk.

I have checked the links to external sites for usefulness. Links to external websites are provided as a convenience. I do not endorse, control, monitor, or guarantee the information contained in any external website. I don’t guarantee that the links are active at all times. Use the links here with the same caution as you would all information on the Internet. This website reflects the thoughts, interests and opinions of its author. They do not explicitly represent official positions or policies of my employer.

Information on this website is subject to change without notice.

Steve Dunbar’s Home Page, http://www.math.unl.edu/~sdunbar1

Email to Steve Dunbar, sdunbar1 at unl dot edu

Last modified: Processed from LATEX source on November 2, 2010