Steven R. Dunbar
Department of Mathematics
203 Avery Hall
University of Nebraska-Lincoln
Lincoln, NE 68588-0130
http://www.math.unl.edu
Voice: 402-472-3731
Fax: 402-472-8466

Note: To best read these pages, you will need the latest version of the Mozilla Firefox browser, with the STIX fonts installed, The Chrome browser will also present a mostly readable version. In a few sections, you will also need the latest Java plug-in, and JavaScript must be enabled. If you use a browser other than Firefox you should be able to access the pages and run the applets. However, mathematical expressions will probably not display correctly. Firefox is currently the only browser that fully supports all of the standards.

_______________________________________________________________________________________________

Rating

_______________________________________________________________________________________________

Section Starter Question

Consider a gambler who wins or loses a dollar on each turn of a fair game with probabilities

p = 1 ∕ 2

and

q = 1 ∕ 2

respectively. Let his initial capital be $10. The game continues until the gambler’s capital either is reduced to 0 or has increased to $20. What is the length of the shortest possible game the gambler could play? What are the chances of this shortest possible game? What is the length of the second shortest possible game? How would you find the probability of this second shortest possible game occurring?

_______________________________________________________________________________________________

Key Concepts

Vocabulary

Mathematical Ideas

Understanding a Stochastic Process

We start with a sequence of Bernoulli random variables,

Y_{1}, Y_{2}, Y_{3}, \dots

where

Y_{i} = + 1

with probability

p

and

Y_{i} = - 1

with probability

q

. We start with an initial value

T_{0}

and set

Y_{0} = T_{0}

for convenience. We define the sequence of sums

T_{n} = \sum_{i = 0}^{n} Y_{i}

. We are interested in the stochastic process

T_{1}, T_{2}, T_{3}, \dots

. It turns out this is a complicated sequence to understand in full, so we single out particular simpler features to understand first. For example, we can look at how many trials the process will experience until it achieves the value

0

a

. In symbols, consider

N = min {n : T_{n} = 0, or T_{n} = a}

It is possible to consider the probability distribution of this newly defined random variable. Even this turns out to be fairly complicated, so we look at the expected value of the number of trials,

D = E [N]

. This is a special case of a larger class of probability problems called first-passage distributions for first-passage times.

Expected length of the game

Note that in the following we implicitly assume that the expected duration of the game is finite. This fact is true, see below for a proof.

Proof. If the first trial results in success, the game continues as if the initial position had been $T_{0} + 1$ . The conditional expectation of the duration conditioned on success at the first trial is therefore $D_{T_{0} + 1} + 1$ . Likewise if the first trial results in a loss, the duration conditioned on the loss at the first trial is $D_{T_{0} - 1} + 1$ .

This argument shows that the expected duration satisfies the difference equation, obtained by expectation by conditioning

D_{T_{0}} = p D_{T_{0} + 1} + q D_{T_{0} - 1} + 1

with the boundary conditions

D_{0} = 0, D_{a} = 0 .

The appearance of the term 1 makes the difference equation non-homogeneous. Taking a cue from linear algebra, or more specifically the theory of linear non-homogeneous differential equations, we need to find the general solution to the homogeneous equation

D_{T_{0}}^{h} = p D_{T_{0} + 1}^{h} + q D_{T_{0} - 1}^{h}

and a particular solution to the non-homogeneous equation. We already know the general solution to the homogeneous equation is $D_{T_{0}}^{h} = A + B {(q ∕ p)}^{T_{0}}$ . The best way to find the particular solution is inspired guessing, based on good experience. We can re-write the non-homogeneous equation for the particular solution as

- 1 = p D_{T_{0} + 1} - D_{T_{0}} + q D_{T_{0} - 1} .

The right side is a weighted second difference, a difference equation analog of the second derivative. Functions whose second derivative is a constant are quadratic functions. Therefore, it make sense to try a function of the form $D_{T_{0}}^{p} = k + l T_{0} + m T_{0}^{2}$ . In the exercises, we show that the particular solution is actually $D_{T_{0}} = T_{0} ∕ (q - p)$ if $p \neq q$ .

It follows that the general solution of the duration equation is:

D_{T_{0}} = T_{0} ∕ (q - p) + A + B {(q ∕ p)}^{T_{0}} .

The boundary conditions require that $A + B = 0$ , and $A + B {(q ∕ p)}^{a} + a ∕ (q - p) = 0$ . Solving for $A$ and $B$ , we find

D_{T_{0}} = \frac{T_{0}}{q - p} - \frac{a}{q - p} \frac{1 - {(q ∕ p)}^{T_{0}}}{1 - {(q ∕ p)}^{a}} .

The calculations are not valid if $p = 1 ∕ 2 = q$ . In this case, the particular solution $T_{0} ∕ (q - p)$ no longer makes sense for the equation

D_{T_{0}} = (1 ∕ 2) D_{T_{0} + 1} + (1 ∕ 2) D_{T_{0} - 1} + 1

The reasoning about the particular solution remains the same however, and we can show that the particular solution is $- {T_{0}}^{2}$ . It follows that the general solution is of the form $D_{T_{0}} = - {T_{0}}^{2} + A + B T_{0}$ . The required solution satisfying the boundary conditions is

D_{T_{0}} = T_{0} (a - T_{0}) .

□

Proof. Pass to the limit $a \to \infty$ in the preceding formulas. □

Illustration 1

The duration can be considerably longer than we expect naively. For instance in a fair game, with two players with $500 each flipping a coin until one is ruined, the average duration of the game is 250,000 trials. If a gambler has only $1 and his adversary $1000, with a fair coin toss, the average duration of the game is 999 trials, although some games will be quite short! Very long games can occur with sufficient probability to give a long average.

Some Calculations for Illustration

Proof that the duration is finite

The following discussion of finiteness of the duration of the game is adapted from [2] by J. Michael Steele.

When we check the arguments for the probability of ruin or the duration of the game, we find a logical gap. We have assumed that the duration

D_{T_{0}}

of the game is finite. How do we know for sure that the gambler’s net winnings will eventually reach

a

0

? This important fact requires proof.

The proof uses a common argument in probability, an “extreme case argument”. We identify an “extreme” event with a small but positive probability of occurring. We are interested in the complementary “good” event which at least avoids the extreme event. Therefore the complementary event must happen with probability not quite

1

. The avoidance must happen infinitely many independent times, but the probability of such a run of “good” events must go to zero.

For the gambler’s ruin, we are interested in the event of the game continuing forever. Consider the extreme event that the gambler wins

a

times in a row. If the gambler is not already ruined (at 0), then such a streak of

a

wins in a row is guaranteed to boost his fortune above

a

and end the game in victory for the gambler. Such a run of luck is unlikely, but it has positive probability, in fact, probability

P = p^{a}

. We let

E_{k}

denote the event that the gambler wins on each turn in the time interval

[k a, (k + 1) a - 1]

, so the

E_{k}

are independent events. Hence the complementary events

E_{k}^{C} = Ω - E_{k}

are also independent. Then

D > n a

at least implies that all of the

E_{k}

with

0 \leq k \leq n

fail to occur. Thus, we find

for all

n

. Hence,

ℙ [D_{T_{0}} = \infty] = 0

, justifying our earlier assumption.

Sources

This section is adapted from [2] with additional background information from [1].

Algorithms, Scripts, Simulations

Algorithm

The goal is to simulate the duration until ruin or victory as a function of starting value. First set the probability

p

, number of Bernoulli trials

n

, and number of experiments

k

. Set the ruin and victory values

r

and

v

, also interpreted as the boundaries for the random walk. For each starting value from ruin to victory, fill an

n \times k

matrix with the Bernoulli random variables. For languages with multi-dimensional arrays each the data is kept in a three-dimensional array of size

n \times k \times (v - r + 1)

. Cumulatively sum the Bernoulli random variables to create the fortune or random walk. For each starting value, for each random walk or fortune path, find the duration until ruin or victory is encountered. For each starting value, find the mean of the duration until ruin or victory. Finally, find a least squares polynomial fit for the duration as a function of the starting value.

Scripts

Problems to Work for Understanding

Reading Suggestion:

References

Outside Readings and Links:

I check all the information on each page for correctness and typographical errors. Nevertheless, some errors may occur and I would be grateful if you would alert me to such errors. I make every reasonable effort to present current and accurate information for public use, however I do not guarantee the accuracy or timeliness of information on this website. Your use of the information from this website is strictly voluntary and at your risk.

I have checked the links to external sites for usefulness. Links to external websites are provided as a convenience. I do not endorse, control, monitor, or guarantee the information contained in any external website. I don’t guarantee that the links are active at all times. Use the links here with the same caution as you would all information on the Internet. This website reflects the thoughts, interests and opinions of its author. They do not explicitly represent official positions or policies of my employer.


$p$	$q$	$T_{0}$	$a$	Probability	Expected
				of Ruin	Duration

0.5	0.5	9	10	0.1000	9
0.5	0.5	90	100	0.1000	900
0.5	0.5	900	1,000	0.1000	90,000
0.5	0.5	950	1,000	0.0500	47,500
0.5	0.5	8,000	10,000	0.2000	16,000,000

0.45	0.55	9	10	0.2101	11
0.45	0.55	90	100	0.8656	766
0.45	0.55	99	100	0.1818	172
0.4	0.6	90	100	0.9827	441
0.4	0.6	99	100	0.3333	162