Steven R. Dunbar
Department of Mathematics
203 Avery Hall
University of Nebraska-Lincoln
Lincoln, NE 68588-0130
http://www.math.unl.edu
Voice: 402-472-3731
Fax: 402-472-8466

Rating

_______________________________________________________________________________________________

Question of the Day

What is the solution of the equation

x_{n} = a x_{n - 1}

where

a

is a constant? What kind of a function is the solution? What more, if anything, needs to be known to obtain a complete solution?

_______________________________________________________________________________________________

Key Concepts

Vocabulary

Mathematical Ideas

Understanding a Stochastic Process

We consider a sequence of Bernoulli random variables,

Y_{1}, Y_{2}, Y_{3}, \dots

where

Y_{i} = + 1

with probability

p

and

Y_{i} = - 1

with probability

q

. We start with an initial value

T_{0}

. We define the sequence of sums

T_{n} = \sum_{i = 0}^{n} Y_{i}

. We are interested in the stochastic process

T_{1}, T_{2}, T_{3}, \dots

. It turns out this is a complicated sequence to understand in full, so we single out particular simpler features to understand first. For example, we can look at the probability that the process will achieve the value

0

before it achieves the value

a

. This is a special case of a larger class of probability problems called first-passage probabilities.

Theorems about Ruin Probabilities

Consider a gambler who wins or loses a dollar on each turn of a game with probabilities

p

and

q = 1 - p

respectively. Let his initial capital be

T_{0}

. The game continues until the gambler’s capital either is reduced to

0

or has increased to

a

. Let

q_{T_{0}}

be the probability of the gambler’s ultimate ruin and

p_{T_{0}}

the probability of his winning. We shall show later that (see also Duration of the Game Until Ruin..)

Proof. After the first trial the gambler’s fortune is either $T_{0} - 1$ or $T_{0} + 1$ and therefore we must have

q_{T_{0}} = p q_{T_{0} + 1} + q q_{T_{0} - 1}

(1)

provided $1 < T_{0} < a - 1$ . For $T_{0} = 1$ , the first trial may lead to ruin, and (1) is replaced by

q_{1} = p q_{2} + q .

Similarly, for $T_{0} = a - 1$ the first trial may result in victory, and therefore

q_{a - 1} = q q_{a - 2} .

To unify our equations, we define as a natural convention that $q_{0} = 1$ , and $q_{a} = 0$ . Then the probability of ruin satisfies (1) for $T_{0} = 1, 2, \dots, a - 1$ . This defines a set of $a - 1$ difference equations, with boundary conditions at $0$ and $a$ . If we solve the system of difference equations, then we will have the desired probability $q_{T_{0}}$ for any value of $T_{0}$ .

Note that we can rewrite the difference equations as

p q_{T_{0}} + q q_{T_{0}} = p q_{T_{0} + 1} + q q_{T_{0} - 1} .

Then we can rearrange and factor to obtain

\frac{q_{T_{0} + 1} - q_{T_{0}}}{q_{T_{0}} - q_{T_{0} - 1}} = \frac{q}{p}

This says the ratio of successive differences of $q_{T_{0}}$ is constant. This suggests that $q_{T_{0}}$ is a power function,

q_{T_{0}} = λ^{T_{0}}

since power functions have this property.

We first take the case when $p \neq q$ . Then based on the guess above (or also on standard theory for linear difference equations), we try a solution of the form $q_{T_{0}} = λ^{T_{0}}$ . That is

λ^{T_{0}} = p λ^{T_{0} + 1} + q λ^{T_{0} - 1} .

This reduces to

p λ^{2} - λ + q = 0 .

Since $p + q = 1$ , this factors as

(p λ - q) (λ - 1) = 0,

so the solutions are $λ = q ∕ p$ , and $λ = 1$ . (One could also use the quadratic formula to obtain the same values, of course.) Again by the standard theory of linear difference equations, the general solution is

q_{T_{0}} = A \cdot 1 + B \cdot {(q ∕ p)}^{T_{0}}

(2)

for some constants $A$ , and $B$ .

Now we determine the constants by using the boundary conditions:

\begin{array}{l} q_{0} & = A + B = 1 \\ q_{a} & = A + B {(q ∕ p)}^{a} = 0 . \end{array}

Solving, substituting, and simplifying:

q_{T_{0}} = \frac{{(q ∕ p)}^{a} - {(q ∕ p)}^{T_{0}}}{{(q ∕ p)}^{a} - 1} .

(Check for yourself that with this expression $0 \leq q_{T_{0}} \leq 1$ as it should be a for a probability.)

We should show that the solution is unique. So suppose $r_{T_{0}}$ is another solution of the difference equations. Given an arbitrary solution of (1), the two constants $A$ and $B$ can be determined so that (2) agrees with $r_{T_{0}}$ at $T_{0} = 0$ and $T_{0} = a$ . (The reader should be able to explain why by reference to a theorem in Linear Algebra!) From these two values, all other values can be found by substituting in (1) successively $T_{0} = 1, 2, 3, \dots$ Therefore, two solutions which agree for $T_{0} = 0$ and $T_{0} = 1$ are identical, hence every solution is of the form (2).

The solution breaks down if $p = q = 1 ∕ 2$ , since then we do not get two linearly independent solutions of the difference equation (we get the solution $1$ repeated twice). Instead, we need to borrow a result from differential equations (from the variation-of-parameters/reduction-of-order set of ideas used to derive a complete linearly independent set of solutions.) Certainly, $1$ is still a solution of the difference equation (1). A second linearly independent solution is $T_{0}$ , (check it out!) and the general solution is $q_{T_{0}} = A + B T_{0}$ . To satisfy the boundary conditions, we must put $A = 1$ , and $A + B a = 0$ , hence $q_{T_{0}} = 1 - T_{0} ∕ a$ . □

We can consider a symmetric interpretation of this gambling game. Instead of a single gambler playing at a casino, trying to make a goal

a

before being ruined, consider two gamblers Alice and Bill playing against each other. Let Alice’s initial capital be

T_{0}

and let her play against adversary Bill with initial capital

a - T_{0}

so that their combined capital is

a

. The game continues until one gambler’s capital either is reduced to zero or has increased to

a

, that is, until one of the two players is ruined.

Proof. The probability $p_{T_{0}}$ of Alice’s winning the game equals the probability of Bill’s ruin. Bill’s ruin (and Alice’s victory) is therefore obtained from our ruin formulas on replacing $p$ , $q$ , and $T_{0}$ by $q$ , $p$ , and $a - T_{0}$ respectively. That is, from our formula (for $p \neq q$ ) the probability of Alice’s ruin is

q_{T_{0}} = \frac{{(q ∕ p)}^{a} - {(q ∕ p)}^{T_{0}}}{{(q ∕ p)}^{a} - 1}

and the probability of Bill’s ruin is

p_{T_{0}} = \frac{{(p ∕ q)}^{a} - {(p ∕ q)}^{a - T_{0}}}{{(p ∕ q)}^{a} - 1} .

Then add these together,and after some algebra, the total is $1$ . (Check it out!)

For $p = 1 ∕ 2 = q$ , the proof is simpler, since then $p_{T_{0}} = 1 - (a - T_{0}) ∕ a$ , and $q_{T_{0}} = 1 - T_{0} ∕ a$ , and $p_{T_{0}} + q_{T_{0}} = 1$ easily. □

Proof. In the game against the infinitely rich adversary, the gambler’s ultimate gain (or loss!) is a Bernoulli (two-valued) random variable, $G$ , where $G$ assumes the value $- T_{0}$ with probability $q_{T_{0}}$ , and assumes the value $a - T_{0}$ with probability $p_{T_{0}}$ . Thus the expected value is

\begin{array}{l} E [G] & = (a - T_{0}) p_{T_{0}} + (- T_{0}) q_{T_{0}} \\ = p_{T_{0}} a - T_{0} \\ = (1 - q_{T_{0}}) a - T_{0} . \end{array}

□

Now notice that if

q = 1 ∕ 2 = p

, so that we are dealing with a fair game, then

E [G] = (1 - (1 - T_{0} ∕ a)) \cdot a - T_{0} = 0

. That is, a fair game in the short run is a fair game in the long run. However, if

p < 1 ∕ 2 < q

, so the game is not fair then our expectation formula says

Proof. Let $a \to \infty$ in the formulas. (Check it out!) □

Some Calculations for Illustration

Why do we hear about people who actually win?

We often hear from people who consistently make their “goal”, or at least win at the casino. How can this be in the face of the theorems above?

A simple illustration makes clear how this is possible. Assume for convenience a gambler who repeatedly visits the casino, each time with a certain amount of capital. His goal is to win 1/9 of his capital. That is, in units of his initial capital

T_{0} = 9

, and

a = 10

. Assume too that the casino is fair so that

p = 1 ∕ 2 = q

, then the probability of ruin in any one year is:

This says that if the working capital is much greater than the amount required for victory, then the probability of ruin is reasonably small.

This much success is reasonable, but simple psychology would suggest the gambler would boast about his skill instead of crediting it to luck. Moreover, simple psychology suggests the gambler would also blame one failure on oversight, momentary distraction, or even cheating by the casino!

Another Interpretation as a Random Walk

Another common interpretation of this probability game is to imagine it as a random walk. That is, we imagine an individual on a number line, starting at some position

T_{0}

. The person takes a step to the right to

T_{0} + 1

with probability

p

and takes a step to the left to

T_{0} - 1

with probability

q

and continues this random process. Then instead of the total fortune at any time, we consider the geometric position on the line at any time. Instead of reaching financial ruin or attaining a financial goal, we talk instead about reaching or passing a certain position. For example, Corollary 3 says that if

p \leq q

, then the probability of visiting the origin before going to infinity is

1

. The two interpretations are equivalent and either can be used depending on which is more useful. The problems below are phrased in the random walk interpretation, because they are more naturally posed in terms of reaching or passing certain points on the number line.

The interpretation as Markov Processes and Martingales

The fortune in the coin-tossing game is the first and simplest encounter with two of the most important ideas in modern probability theory.

We can interpret the fortune in our gambler’s coin-tossing game as a Markov process. That is, at successive times the process is in various states. In our case, the states are the values of the fortune. The probability of passing from one state at the current time

t

to another state at time

t + 1

is completely determined by the present state. That is, for our coin-tossing game

The most important property of a Markov process is that the probability of being in the next state is completely determined by the current state and not the history of how the process arrived at the current state. In that sense, we often say that a Markov process is memory-less.

We can also note the fair coin-tossing game with

p = 1 ∕ 2 = q

is a martingale. That is, the expected value of the process at the next step is the current value. Using expectation for estimation, the best estimate we have of the gambler’s fortune at the next step is the current fortune:

This characterizes a fair gain, after the next step, one can neither expect to be richer or poorer. Note that the coin-tossing games with

p \neq q

do not have this property.

In later sections we have more occasions to study the properties of martingales, and to a lesser degree Markov processes.

Sources

This section is adapted from W. Feller, in Introduction to Probability Theory and Applications, Volume I, Chapter XIV, page 342, [1]. Some material is adapted from [3] and [2]. Steele has an excellent discussion at about the same level as I have done it here, but with a slightly more rigorous approach to solving the difference equations. He also gives more information about the fact that the duration is almost surely finite, showing that all moments of the duration are finite. Karlin and Taylor give a treatment of the ruin problem by direct application of Markov chain analysis, which is not essentially different, but points to greater generality.

_______________________________________________________________________________________________

Problems to Work for Understanding

Reading Suggestion:

References

Outside Readings and Links:

I check all the information on each page for correctness and typographical errors. Nevertheless, some errors may occur and I would be grateful if you would alert me to such errors. I make every reasonable effort to present current and accurate information for public use, however I do not guarantee the accuracy or timeliness of information on this website. Your use of the information from this website is strictly voluntary and at your risk.

I have checked the links to external sites for usefulness. Links to external websites are provided as a convenience. I do not endorse, control, monitor, or guarantee the information contained in any external website. I don’t guarantee that the links are active at all times. Use the links here with the same caution as you would all information on the Internet. This website reflects the thoughts, interests and opinions of its author. They do not explicitly represent official positions or policies of my employer.

$p$	$q$	$T_{0}$	$a$	Prob of Ruin	Prob of Success	Exp Gain	Duration
0.5	0.5	9	10	0.1000	0.9000	0	9
0.5	0.5	90	100	0.1000	0.9000	0	900
0.5	0.5	900	1,000	0.1000	0.9000	0	90,000
0.5	0.5	950	1,000	0.0500	0.9500	0	47,500
0.5	0.5	8,000	10,000	0.2000	0.8000	0	16,000,000

0.45	0.55	9	10	0.2101	0.7899	-1	11
0.45	0.55	90	100	0.8656	0.1344	-77	766
0.45	0.55	99	100	0.1818	0.8182	-17	172
0.4	0.6	90	100	0.9827	0.0173	-88	441
0.4	0.6	99	100	0.3333	0.6667	-32	162