Reading The Fine Print: Margin of Error
by Glenn Iwata
Poll Popularity
Everybody conducts surveys these days. Open up the newspaper and you can
read opinion polls on everything from public policy to deodorant. With
the proliferation of these surveys, there occasionally will be some fine
print below the reported results - a.k.a., the margin of error. Quite
often the margin of error is more important than the splashy headlines
that tout the results of the poll. This article defines the margin of
error and explains how it is calculated.
Margin of Error Defined
The margin of error is simply a measure of how "precise" the data are.
The margin of error is a necessary tool because few organizations have
the time or money to survey the entire population on a particular subject.
Rather, "samples" of the population are taken using various surveying
techniques. As a result, it is necessary to know how precisely the results
of the sample reflect true feelings of the entire population.
For Example...
To explain this concept more clearly, let's say for a moment that you
received a report that states, "50% of the Phoenix-area residents remembered
seeing McDonald's ads on television." The fine print of this report further
states that there is a "margin of error of plus or minus 10% at the 95%
level of confidence."
Translation:
If 100 surveys were completed with Phoenix residents, the true percentage
answer (i.e., if every Phoenix resident completed the survey) would fall
between 40% and 60% in 95 of the 100 surveys. Some people mistakenly say,
"The sample is 95% accurate." But remember, the margin of error is a measure
of precision, not accuracy. Ultimately, the bottom line is that the 10%
margin of error is a pretty wide margin. The real answer could just as
likely be 40%, or 60%, or any other number in between. The best estimate
of the sample is that 50% of the population saw the commercials.
Calculating Margin of Error
As you may (or may not) remember from your "Introduction to Statistics"
class, the formula for calculating the margin of error is:
The formula involves three basic parts:
1. The amount of variability within the sample
In this formula, p is an estimate of the percentage of respondents answering
a particular question. Our example assumed that 50% of the respondents
saw the McDonald's commercials (p=.50, 1-p=.50). When reporting the margin
of error for an entire study, a 50% average response is used because it
is the most "conservative" estimate.
2. The degree of precision
A "standard" level of precision is a 95% confidence level (which was described
above). This translates into 1.96 (two standard deviations for you statistic
gurus) for this formula.
3. The sample size
This is the number of respondents (n) that answered the question.
For our example, let's assume that 90 people completed the survey.
Applying the Margin of Error Formula
If we apply these three parts to our McDonald's commercial example, the
10% margin of error is calculated in the following manner:
There is a fourth element that should be added when the population being sampled is "small," but we will save that for your next Statistics 101 lesson.
