Confidence in Sampling
Over the years many people have questioned statistical analysis and its use of sampling. Many find sampling questionable because statisticians often make inferences about populations that are based on relatively small samples. More recently there has been much debate about the potential use of sampling to simplify national census procedures. This article explains how sampling, when used with careful guidelines, can be accurate and useful.
"Now let there be no doubt about it, the sample is a treacherous device for drawing conclusions about how the world works. Statisticians may have problems with it, sometimes drawing broad conclusions on the basis of a very limited sample. Ordinary people have lots of problems with it — though they are often unaware of the fact that they may be over-generalizing. The major difference between the statistician and the untrained person is that the statistician knows more clearly what these problems are and how to cope with them."1
Sampling is ingrained into every day of our lives. We take samples of many things individually and then infer judgments as a result of this "research." For example, you meet a person and, after three minutes of conversation, you decide you do not like him. We often make broad, and usually unfair, generalizations about a person's character on the basis of brief contact. Or, what about when you visit the doctor's office and they take a sample of blood to run tests. From just a few drops — a small sample — they are able to make definite conclusions that could dramatically affect your life.
So what are the main strategies that good statisticians use to deal with the uncertainty associated with sampling? There are four main ones we will discuss briefly in this article.
-
First, know the variability within the population being sampled. The more homogenous the population is, the smaller the sample need be to obtain reliable results. However, when the survey is about social issues or qualities, there is generally going to be more variability in the population and a larger sample would be required.
-
The size of the sample itself influences its reliability. The bigger the sample, assuming it is a random sample, the more confidence we can have in its reliability. However, this too has its limitations. Achieving confidence above 95% requires dramatically larger sample sizes, which, in turn, increases research costs.
-
Randomizing a sample can also help to ensure that it is representative of the main population. In some cases, however, a random sample is not appropriate — but this is a subject for another article.
-
And finally, higher levels of precision require larger samples. Intuition tells us that the more people we survey the more accurate the overall results will be. Therefore, if the results of the research will have a large impact on specific important decisions, a good statistician would recommend a larger sample.
Although making inferences about populations from small samples may seem risky, with proper procedures and knowledgeable statisticians reliable data can be obtained.
1. Cuzzort, R. P. and Vrettos, J. S., The Elementary Forms of Statistical Reason, St. Martin's Press, 1996, p. 19.
