Analyze That
The Skinny on Weighting
by
Stephanie Apostol with contributions by Glenn Iwata and Ted Apostol

Now that holidays are over and some of us are thinking about shedding a few pounds for the New Year, WestGroup thought it appropriate to give you … "The Skinny on Weighting."

We understand that many data manipulation techniques that seem simple to market research professionals are not so easy for those who don't spend their lives getting cozy with data. One such technique is weighting. This is a method we frequently use and find to be an ever more inviting solution to the increasing cost of achieving demographic quota goals.

Weighting is the process by which data are adjusted to better reflect your target population. Weighting is a step during the data management process that provides a greater or lesser impact to individual respondents' answers based on his/her particular demographic or psychographic categories. Instead of each survey participant counting as a single respondent, a person in an underrepresented group, like grandmothers who rap to Eminem, might count more (e.g., given a weight of two instead of one) than someone's responses from an overrepresented group, like retirees in Sun City who love to talk on the phone (e.g., given a weight of .25 instead of one).

Why weight? Weighting the data will counter effects of differential refusal rates, falling short on particular quotas, or to correct for any over-sampling of minority populations. We need to weight the sample if the responses show that a particular group, for example, younger people or those living in a particular area, are underrepresented in the sample. If this is not carried out, then the results may not properly reflect the views of the population being considered. It serves the purpose of providing data that look like the population it represents.

Benefits of Weighting
The benefits of weighting are primarily driven by convenience and financial considerations. Weighting allows you to reflect your population exactly, without the expense of meeting strict quotas. This is important because some groups that are difficult to reach, such as males under the age of 25, might make data collection costs soar.

Drawbacks of Weighting
Any time you weight data, you are penalized in terms of statistical accuracy. A sub-group that is given more weight appears to be larger and more statistically reliable than it actually is.

Another drawback is the unknown. The reason for the underrepresentation of a sub-group given more weight could skew results for that group. For example, young males tend to be underrepresented in strictly random samples because they are harder to reach by telephone and less cooperative than other respondents. If they are hard to reach due to something that makes them different (i.e., high reliance on cell phones), those who do respond may be very different from those who do not.

Recommendation
Weighting is a great solution to keeping research costs reasonable. We recommend weighting when the cost of quota control is too high. Clients and their research partners should always make the weighting decision together, fully discussing the advantages and disadvantages.