Steven R. Dunbar
Department of Mathematics
203 Avery Hall
University of Nebraska-Lincoln
Lincoln, NE 68588-0130
Probability Theory and Stochastic Processes
Steven R. Dunbar
An Analytic Model for Coin Tossing
Mathematicians Only: prolonged scenes of intense rigor.
What are the axioms for a probability space? What are the assumptions about coin-ﬂips or successive Bernoulli trials that make the process a probability space?
converges uniformly on bounded intervals to
the Rademacher functions are .
The following is due both to Vieta in 1593 and also to Euler.
Remark. The function is often called the sinc function. The cosine product deﬁnition shows that while is undeﬁned at . However, so is continuous at . In the sequel the implicit assumption is that is at even though technically the expression is undeﬁned. The sinc function , also called the “sampling function”, is a function that arises frequently in signal processing and the theory of Fourier transforms. The full name of the function is “sine cardinal”, but it is commonly referred to by its abbreviation, “sinc”.
Remark. The following proof is remarkably elementary for such an unusual identity. The proof combines repeated application of the half-angle identity for the cosine with an elementary limit of the sine function. The proof of uniform convergence uses the Cauchy criterion. The veriﬁcation of the criterion uses an elementary bound on the cosine function from the series expansion.
and then let . Details are left to the reader.
A special case is the basis for an unusual formula involving due to Vieta in a book published in 1593. First, a simple trigonometric lemma.
so that the assumption is that
establishing the induction.
Proof. Set and apply the lemma to the inﬁnite product formula for from the theorem. □
A numerical evaluation of successive ﬁnite products is in Table 1, illustrating the convergence.
Every with has a unique binary expansion,
using the convention that terminating expansions have all digits equal to from a certain point. For example, write
With the convention about terminating expansion, the graphs of , , , …are as in Figure 2.
It is more convenient to use the Rademacher functions deﬁned by , with graphs as in Figure 2. Note the similarity to the random variables and .
In terms of the Rademacher functions, the binary expansion (1) becomes
Remark. Note that an alternative description of the Rademacher functions is
In this sense, the Rademacher functions are a discretized version of the sine functions. The Rademacher function expansion follows directly from binary expansion and not from an orthogonal basis.
Remark. This is remarkable and unusual, since it says an integral of a product is a product of integrals. The proof combines the expansion of in Rademacher functions, elementary integration, the expression of and in terms of complex exponentials, and Vieta’s formula.
Remark. On the other hand, from an advanced probability point of view, it may not be so remarkable after all. Consider as a random variable over the probability space . Although the left side is a product of exponentials, it could be written as the exponential of a sum of random variables. The left side is then the characteristic function (or re-scaled Fourier transform) of the sum of random variables. By a well-known correspondence, the characteristic function of a sum is equal to the product of individual characteristic functions. Characteristic functions transform questions about sums of random variables and convergence of random variables into analytic questions of products and pointwise convergence of functions. In this way, this theorem points the way to the analytic model of coin-ﬂipping below.
Consider the function . It is a step function that is constant on the intervals for . The values of the function are . Every sequence of length of s and s corresponds to exactly one interval . There are such intervals and the sequence of corresponds to the sequence of subintervals of length with that is in. Then
and the lower limit on the summation indicates that the summation over each of the possible subsequences of . By turning a sum of powers into a product
Putting together (2) and (3)
where the proof of the last equality is the same as in step 2 of the proof of Theorem 3.
Now set to get
converges uniformly in to by the Weierstrass M-criterion. Therefore we can interchange limit and integral to see that
This is a diﬀerent proof of Vieta’a formula connecting the formula to binary representations of real numbers in .
Let be the length, or more formally, the measure of the subset of . If is a sequence of ’s and ’s then
Example. Let , , . Then , , . Consider the set . Then
This gives another way to rewrite the proof of (4) that is basically the same as before:
The following correspondence gives an analytic model of successive Bernoulli trials. That is, consider the following probability scenario. A fair coin is successively tossed times, with independence of tosses, coming up Heads or Tails on each toss. Use Table 2 to interpret this physical probability experiment analytically
|Coin tossing||Analytic Model|
|Event||set of ’s in|
|Probability of an event||Measure of the corresponding set of ’s|
Example. Consider an analytic model for the simplest probability problem for Bernoulli trials: Find the probability that in independent tosses of a fair coin, exactly will be heads. Using the table to translate the problem to analytic terms, the problem becomes: Find the measure of the set of ’s such that exactly of the numbers , , …, are equal to .
To solve this problem start by noticing that having exactly of of the being means that are , so that
Second, notice that
for and so
will equal on the set of ’s satisfying the condition (5), and is equal to otherwise. (Pay careful attention to the variable of integration.) Therefore
The interchange of order of integration is usually justiﬁed by Fubini’s Theorem, but that is not actually necessary here, since is a step function. Now use (3) on the inner integral with to obtain
Evaluating this integral is in the problems, the result of the evaluation gives
The natural mathematical modeling assumption for probability is that events that seem unrelated are probabilistically independent. That is, the joint probability of unrelated events is the product of the individual probabilities. The product rule is not a mathematical necessity. Rather it is a modeling rule based on experiment and practical experience, justifying the multiplication of probabilities. Thus, probabilistic independence is an intuitive notion with the sense that the multiplication rule is applicable and useful.
In a landmark 1909 paper, “Sur les probabilités dénombrables et luer applications arithmétiques” E. Borel showed that binary digits, or equivalently the Rademacher functions, are independent in the sense that
This observation gives well-deﬁned mathematical objects to which the postulates of probability theory apply directly.
The application of the postulates of probability to Rademacher functions eliminates casting probability in terms of coins and tosses. All of the previous proofs of theorems for coin-tossing have equivalents using Rademacher functions, or equivalently binary digits, with Table 2. The Weak Law of Large Numbers, the direct Borel-Cantelli Lemma and the Strong Law of Large Numbers serve as examples.
Remark. Consider the example in Figure 4.
Each of the integrals over subintervals , , and are like a rescaled version of , so the entire integral is . The proof expands that idea using induction on the number of factors in the product.
the summation becomes
Remark. The proof is essentially the standard one using Chebyshev’s inequality expressed directly in terms of the measure and integral of Rademacher functions.
Remark (Borel-Cantelli Direct Half). The proof is a direct translation of the probabilistic proof of the direct Borel-Cantelli Lemma, using an analytic version of Markov’s inequality.
Remark. This proof resembles the third proof of the Strong Law in Strong Law of Large Numbers.. The proof substitutes the analytic form of the direct half of the Borel-Cantelli Lemma to show the convergence almost everywhere.
since the only terms that contribute positive values are the fourth powers and the pairs of squares.
almost everywhere, and so
Remark. Recall that and that is the th bit in the binary expansion of . Then the Strong Law of Large Numbers implies that for almost every
In other words, almost every number has asymptotically the same number of ’s and ’s. That is, we say that the binary expansion is normal.
This section is adapted from: Statistical Independence in Probability, Analysis, and Number Theory by Mark Kac, 1959, Mathematical Association of America, pages 1–35. Problems 2 and 3 are from the same source. The Cauchy criterion for uniform convergence is Theorem 7.8, page 134 in Principles of Mathematical Analysis second edition by W. Rudin, McGraw-Hill, 1964. The remarks about the sinc function are from Mathworld: Sinc function.
where or . Prove that is independent of .
Plot the functions , , and so on, and show that they are independent. This is the basis for an analytic model of the unfair coin. See the next problem.
I check all the information on each page for correctness and typographical errors. Nevertheless, some errors may occur and I would be grateful if you would alert me to such errors. I make every reasonable eﬀort to present current and accurate information for public use, however I do not guarantee the accuracy or timeliness of information on this website. Your use of the information from this website is strictly voluntary and at your risk.
I have checked the links to external sites for usefulness. Links to external websites are provided as a convenience. I do not endorse, control, monitor, or guarantee the information contained in any external website. I don’t guarantee that the links are active at all times. Use the links here with the same caution as you would all information on the Internet. This website reﬂects the thoughts, interests and opinions of its author. They do not explicitly represent oﬃcial positions or policies of my employer.
Information on this website is subject to change without notice.
Steve Dunbar’s Home Page, http://www.math.unl.edu/~sdunbar1
Email to Steve Dunbar, sdunbar1 at unl dot edu
Last modiﬁed: Processed from LATEX source on June 18, 2018