Writing Assignment, due Monday, November 21:

First, some background!

Suppose you have a survey that asks a single yes/no question. We know that of the whole population, some fraction of the people, call this fraction P, will feel that the right answer to the question is Yes.

In order to measure what that fraction P is, polling companies like Gallup take a random sample. Let n be the sample size. The bigger n is the more reliable the measurement is, but the more expensive it is to do the sample.

The math behind the reliability is this. Suppose you could check every possible sample of n people. For each sample you find the proportion p^, called "p hat", of people in that sample that answered Yes. Suppose you could make a histogram of all of this data, showing how many of the samples there were for each possible value of p^. The fact is that the histogram would be very close to a normal curve (i.e., a bell shaped curve). The peak of the curve would occur for the value of p^ equal to P. In other words, if you were to average p^ for all of the possible samples of n people, the average would be P, which is what you want to know.

Unfortunately, there is no way to check all possible samples of n people, since there usually is no practical way to survey everybody in the whole population. But we know that there is a 95% chance that the value of p^ for any single sample of n people will be within 2 standard deviations of the population average, P. Thus by taking a single random sample (it must be random for this to be true), the value p^ you get for that sample has a 95% chance of being within 2 standard deviations of P.

The formulas are as follows:
1 standard deviation for the number of people who say Yes is equal to the square root of (nP(1-P)).
To get the standard deviation for the FRACTION who say Yes you just have to divide the previous formula by n. This simplifies to:
1 standard deviation for the fraction of people who say Yes is equal to the square root of (P(1-P)/n).
Since we don't know what P is, we estimate this standard deviation by using p^ in place of P. When we do this statisticians call the result the standard error, just to distinguish it from what you get when you use P to get the exact value. The standard error is denoted s. Thus:
s = the square root of (p^(1-p^)/n).

The result is that we expect P to be between p^ - 2s and p^ + 2s for 95% of the samples we take (to say it differently, whenever we take a random sample, we expect there to be a 95% chance that the actual value of P is within 2s of the value p^ we measure with the sample). This is true regardless of the size of the whole population. In particular, even if the whole population is huge, you still get a measurement of P expected to be 95% reliable (to within 2s) by taking a random sample of size n. The size of the whole population is not a factor in the level of reliability, so you really can get reliable results even though the size of the sample n might be a minuscule fraction of the whole population. In particular, even though you survey only 1000 people out of a whole population of 100 million, you do in fact have a pretty good idea of how all of the people feel who were not in the survey: you're 95% sure that the fraction P of them that would answer yes is within 2s (where s is given by the formula above) of the fraction p^ you found with the survey. (This 2s is called the margin of error for the survey.)

One way to explain the fact that you can be pretty sure you know something about all of the people who were not in the survey by asking a very small fraction of the whole population is to compare it to how you decide whether soup is too salty. Whether it's a cup of soup, a bowl of soup or the whole pot of soup, you just need a taste to tell if it's too salty. To tell if the whole pot is too salty you don't need to taste more than what is needed to check whether a cup or a bowl is too salty.

So here is the writing assignment: suppose you bring to a 6th grade class a newspaper article that says a survey of 1100 randomly selected people found that 45% of all Americans felt a certain way on a certain topic, with a margin of error of plus or minus 3%. How would you explain what this means? Your write up should explain why it does not mean that it is certain that the percentage of all Americans who feel that way is between 42% and 48%. It should also discuss how you would explain that asking only 1100 people is sufficient to give you a reliable measurement of how the other millions of Americans who were not in the survey feel, and what that level of reliability is.