Download the full PDF version of this issue (181K).
(Viewable with Acrobat Reader from Adobe.)
Research 101: "Did the needle move"...Researching
Change
Dave Maddox
In conference rooms, offices and boardrooms everywhere, in one form or another, one of the most common problems facing businesses is 'How do we know 'IT' is working?' 'IT' is frequently a marketing-related issue, whether it be advertising, public relations or promotions.
A prime example of 'IT' might be a new advertising campaign that has been launched, and the advertiser wants to evaluate the success or failure of the campaign.
The most common (and effective) method for evaluating these programs is with a pretest/ post-test quantitative study. In a pretest/ post-test study, quantitative research is conducted just prior to launching a new campaign, then replicated immediately after the campaign has run its course. In this manner, the findings of the follow-up (posttest) research are compared against the 'baseline' from the pre-test to evaluate any changes that may have occurred.
In a real-world setting, other variables can impact attitudes or behaviors in addition to advertising; the key is to control as many of those variables as possible, or to take those beyond our control into account in analyzing the results.
Key Variables
There are many variables by which to evaluate changes that occur between pre- and post-test phases, but the following are the most common (and useful):
Unaided Awareness. Unaided, or 'top-ofmind' awareness, is simply the proportion of respondents who can name a product, company, candidate, etc. without prompting. For example, we may ask "tell me the names of homebuilders in the Phoenix area? Any others?"
Aided Awareness. Once a respondent exhausts all their responses to the unaided question, it is common to then prompt for awareness of key brands and competitors. Using the same example, we might ask "Now I'm going to read a short list of homebuilders, and I'd like you to tell me whether you have heard of each." The inclusion of competitors when testing awareness and familiarity allows us to analyze the relative position of the client within the marketplace.
Familiarity. If desired, another level of awareness can be incorporated into the prompted questions. Instead of simply asking whether a respondent has heard of a particular brand and competitors, we can ask how this case, homebuilders), using some type of scale. This approach provides richer data than a simple yes/no awareness finding.
Image. Time and budget permitting, another common topic for this type of research is to examine the relative image of a brand across different variables. In our homebuilder example, we may ask the respondent to rate the builder on their reputation, product quality, customer service, or many other topics. Changes in these dimensions are especially meaningful if similar messages are part of the advertising being tested.
Advertising recall. In most cases, the success of a campaign is evaluated by an indirect variable, such as awareness, traffic counts, or (ultimately) increased sales. In those cases where specific media are being evaluated, it is simple enough to ask respondents whether they have seen or heard relevant advertising during the test period, and which type of advertising it was (TV, radio, outdoor, etc.) We also ask those aware of advertising for feedback on the advertising message, to further evaluate its effectiveness.
Demographics. In almost every case, we also will ask certain questions of respondents regarding their age, affluence, education, and other factors to help analyze the findings across different demographic categories. The actual categories to be studied will vary from client to client, but the purpose is the same: to evaluate the success of the campaign among specific target audiences, in addition to the marketplace as a whole.
It's easier than you think
The most common barrier to this type of research is the lack of a proper
baseline. Either the advertising has already begun (or is about to), or
some other variable might adversely impact the results. We work with many
clients on ongoing research of this type, whether through a few simple
questions on WestTrack, or custom tracking studies. Some clients conduct
regular monthly, quarterly or semi-annual tracking studies, testing those
variables listed above and many others. This proactive approach allows
them to quickly implement new campaigns, with the confidence that they
will have a basis for evaluating the campaign's success.
For further information, contact Dave
Maddox at 602-707-0050.
