# Topics in Probability Theory and Stochastic Processes Steven R. Dunbar

### Key Concepts

1. Markov’s Inequality: Let be a random variable taking only non-negative values. Then for each
2. Chebyshev’s Inequality: Let be a random variable. Then for
3. Weak Law of Large Numbers For

and the convergence is uniform in .

4. Let be a real function that is defined and continuous on the interval . Then

as .

### Vocabulary

1. The family of polynomials

are called the Bernstein polynomials.

### Mathematical Ideas

This section is adapted from: Heads or Tails, by Emmanuel Lesigne, Student Mathematical Library Volume 28, American Mathematical Society, Providence, 2005, Chapter 5

#### Proof of the Weak Law Using Chebyshev’s Inequality

Proposition 1 (Markov’s Inequality) Let be a random variable taking only non-negative values. Then for each

Proof 1

Q.E.D.

Proposition 2 (Chebyshev’s Inequality) Let be a random variable. Then for

Proof 2

This immediately follows from Markov’s inequality applied to the non-negative random variable . Q.E.D.

Theorem 1 (Weak Law of Large Numbers) For

and the convergence is uniform in .

Proof 3

The variance of the random variable is . Rewrite the probability as the equivalent event:

By Chebyshev’s inequality

Since , the proof is complete. Q.E.D.

Note that the proof demonstrates that

uniformly in .

#### Bernstein’s Proof of the Weierstrass Approximation Theorem

Theorem 2 Let be a real function that is defined and continuous on the interval . Then

as .

Proof 4

1. Fix . Since continuous on the compact interval it is uniformly continuous on . Therefore there is an such that if .
2. The expectation can be expressed as a polynomial in :
3. By the Weak Law of Large Numbers, for the given , there is an such that
4. Apply the triangle inequality to the right hand side and express in terms of two summations:

Not the second application of the triangle inequality on the second summation.

5. Now estimate the terms:
6. Finally, do the addition over the individual values of the probabilities over single values to re-write them as probabilities over events:
7. Now apply the Weak Law to the second term to see that:

This shows that can be made arbitrarily small, uniformly with respect to , by picking sufficiently large.

Remark 1 The family of polynomials

are called the Bernstein polynomials. The Bernstein polynomials have several useful properties:

1. for .

Corollary 1 The polynomial uniformly approximating the continuous function on the interval is

### Problems to Work for Understanding

1. Heads or Tails, by Emmanuel Lesigne, Student Mathematical Library Volume 28, American Mathematical Society, Providence, 2005, Sections 1.2 and Chapter 4.
2. An Introduction to Probability Theory and Its Applications, Volume 1, William Feller, John Wiley and Sons, 1968, ISBN-13: 978-0471257080, pages 228-247.
3. Probability, by Leo Breiman, SIAM: Society for Industrial and Applied Mathematics; Reprint edition (May 1, 1992), Philadelphia, 1992, ISBN-13: 978-0898712964.