The passage from discrete random walks to continuous
stochastic processes, from the probability point of view and
the partial differential equation point of view.
- A
**diffusion process**, or a
**diffusion** for short, is a Markov process
for which all sample functions are continuous. (From Karlin
and Taylor, page 30.) Also, a solution to a stochastic
differential equation. (From Baxter and Rennie, page 61.)

The following discussion is taken from W. Feller, in
*Introduction to Probability Theory and Applications,
Volume I*, Chapter XIV, page 354. Consider a random walk
starting at the origin. The -th step takes the particle to
the position
, the sum of n independent,
identically distributed Bernoulli random variables
assuming the values , and with probabilities and
respectively. Then recall that the mean of a sum of random
variables is the sum of the means:
and the variance of a sum of *independent* random
variables is the sum of the variances:
Now suppose we want to display a motion picture of the random
walk moving left and right along the x-axis. We want the
motion picture to display 1 million steps and be a reasonable
length of time, say 1000 seconds, between 16 and 17 minutes.
This fixes the time scale at a rate of one step per
millisecond. What should be the spatial scale in order to get
a good sense of the random walk? For this question, we use a
fixed unit of measurement, say centimeters, for the length of
the screen and the individual steps. We are then concerned
with
, where stands for the length of the
steps. Now
and
We want to be large (about 1 million) and we want
to be comparable to the screen size, (say 30cm), so
will be small
cm mm!).
But then
will be so small as to be
indistinguishable
cm).
We will not see the random
variations! We want to be able to see the drift induced by
the difference , and to be consistent with the variance
requirement this will only be possible if is comparable
in size to . Since
, then likewise
, while , so .
The analytical formulation of the problem is as follows. To
every choice: let be the size of the individual steps,
let be the number of steps per unit time. We ask what
happens in the limit where
,
, and
in such a manner that:
and

When we are in possession of an explicit expression for the
relevant probabilities, we can pass to the limit directly.
The method is of limited scope however, since it does not
easily lend itself to generalizations. Furthermore, the limit
manipulations on the explicit expression get more and more
complicated. Let
be the probability that the -th step is at position . In
our accelerated random walk, the -th step takes place at
time and the position is at
. We are
interested in the probability of finding the particle at
given instant and in the neighborhood of a given point ,
so we investigate the limit of as ,and
.
Now think about it for a few minutes and you will see that
the random walk can only reach an even-numbered position
after an even number of steps, and an odd-numbered position
after an odd number of steps. Therefore, is an
integer and we reach position number at timestep if the
particle takes steps to the right. This happens with
probability
From the Central Limit Theorem
Since is the probability of finding
between
and
, and since this interval
has length
we can say that the ratio
measures locally the probability per unit
length, that is the probability density. The last relation
above implies that the ratio tends to
It follows by the definition of integration as the sums of
quantities representing densities times geometric lengths or
areas, that sums of probabilities can be approximated
by integrals and the result may be restated to the effect
that
The integral on the right may be expressed in terms of the
standard normal distribution function and in fact is only a
notational variant of the Central Limit Theorem for
the binomial distribution.
Another fruitful method is to start from the difference
equations governing the random walk, and then pass
to the differential equation in the limit. We can then
obviously generalize the differential equations, and find out
that the differential equations govern well-defined
stochastic processes depending on continuous time. Since
differential equations have a powerful theory and many
available tools to manipulate, transform and solve them, this
method turns out to be useful in the long-run. Consider
the position of the particle in the random walk at the th
and -st trial, thorugh a first step analysis
it is obvious that the probabilities
satisfy the difference equations:
Now in the limit as
and
,
will be the sampling of the function at time
intervals , so that , and space intervals so that
. That is the function should be an
approximate solution of the difference equation:
We assume is a smooth function so that we can expand
in a Taylor series at any point. Using the first order
approximation in the time variable on the left, and the
second-order approximation on the right in the space
variable, we get (after canceling the leading terms )
In our passage to the limit, the omitted terms of higher
order tend to zero, so may be neglected. The remaining
coefficients are already accounted for in our limits and so
the equation becomes:
This is a special *diffusion equation*, more
specifically, a diffusion equation with convective or drift
terms, also known as the Fokker-Planck equation for
diffusion. It is a relatively standard problem to solve the
differential equation for and therefore, we can find
the probability of being at a certain position at a certain
time, relatively easily. The diffusion equation can be
immediately generalized by permitting the coefficients and
to depend on , and . Furthermore, the equation possesses
obvious analogues in higher dimensions and all these
generalization can be derived from general probabilistic
postulates. We will ultimately describe stochastic processes
related to these equations as *diffusions*.
One can verify that
is a solution of the diffusion equation, so we reach the same
probability distribution for .
*Introduction to Probability Theory and Applications*,
Volume I</em>, by W. Feller, Chapter XIV, page 354, J Wiley
and Sons, New York

This document was generated using the
**LaTeX**2`HTML` translator Version 2002-2-1 (1.70)

Copyright © 1993, 1994, 1995, 1996,
Nikos Drakos,
Computer Based Learning Unit, University of Leeds.

Copyright © 1997, 1998, 1999,
Ross Moore,
Mathematics Department, Macquarie University, Sydney.

The command line arguments were:

**latex2html** `intro_diffusion.tex`

The translation was initiated by Steven Dunbar on 2007-10-24

*Steven R. Dunbar*

Department of Mathematics and Statistics

University of Nebraska-Lincoln

Lincoln, NE, 68588-0323 USA

email: sdunbar@math.unl.edu

Steve Dunbar's Home Page
*
*