Ad Awareness and Effectiveness

The number of ways to advertise spans the gamut from very expensive Super Bowl TV ads to simple tweets. Clearly both of these extremes will be seen by vastly different audiences. However, even inexpensive advertising has the potential to go viral if the right conditions exist. By extension, monitoring and improving ad awareness and effectiveness may help small organizations reach large number of potential customers.

It may be tempting to solicit feedback from those of interest to the client by simply asking if respondents recall and/or have acted upon a particular advertisement. While this would provide some information on the awareness of an advertisement, it provides only a limited understanding of how the ad is perceived in the marketplace in general; how has the client’s advertising campaign fared compared to their competitors? There may also be “piggyback” effects at play where respondents are confusing the client’s advertisement with another, similar advertisement. This simple analysis does not provide any of this insight.

A better method is to take respondents down an ever-narrowing path of questions ranging from recall to action:

In this particular case, respondents were asked if they had seen any advertisements within the client’s market. Naturally, this would include any advertisements by the client or their competitors. Next, respondents were shown a number of slogans and asked if they had heard or seen them. Finally, several of the client’s specific advertisements were shown to respondents and those who recalled the ads were asked if they had discussed them, which was the client’s ultimate goal. This simple analysis tells the client that while their specific advertisements may not be very memorable, their slogans certainly resonate.

This analysis is very simple and as a result, has some drawbacks. First, it does not tell you any information about potential acquiescence bias – survey respondents’ tendency to agree with questions, in this case, saying that they’ve heard a slogan or ad when really they have not (but perhaps think “they should have”). Additionally, the only competitive intelligence is shown in the second column where the client sees how often any ads similar to theirs are being recalled. It does not tell the client how they stack up against specific competitors. If it turns out, for example, that specific competitors’ ads are less memorable than the client’s, then perhaps little or no action is needed, even though the client’s ads are not memorable, but that information is not available from this analysis.

Respondents could be asked the same battery of questions several times for any number of competitors. However, this will be challenging for respondents, leading to respondent fatigue and drop-off. Additionally, the analyst still wouldn’t have a method to estimate potential acquiescence bias.

Rather than make the survey and analysis complex, a simple solution is to ask only one set of questions not only about the client’s specific ads or slogans, but also about competitors’ and fictitious ads or slogans. Keeping the survey simple reduces respondent burden. Keeping the analysis simple means that even if the client’s budget is limited, they can collect valuable competitive information. In this particular case, we specifically asked survey respondents if they recalled the client’s slogans, competitors’ slogans, and fictitious slogans, with the results in the following chart.

The results of this analysis revealed these insights:

1. Acquiescence bias: Roughly 10% of survey respondents will claim to definitely have heard a slogan and another quarter will think they may have heard it, even if it is fictitious.
2. Awareness context: The client’s 3rd slogan is not particularly memorable as it performs no better than one of the fictitious slogans.
3. Competitive intelligence: The client’s other two slogans are about half as memorable as their competitors’ slogans.

In this case, the client’s advertising budget was a fraction of their competitors’. The fact that two of the client’s slogans are recognized by about half as many people as their competitors’ was very encouraging to the client. However, it also provides further context for the specific advertisement recall shown in the waterfall chart. If 10% of survey respondents claim to have heard a slogan that is fictitious, then likely a similar percentage would claim to have seen a specific ad even if they had not. Ultimately, this means that many of the 8% shown in the waterfall chart who claim to have seen a specific ad likely did not and the ads are even less memorable than the waterfall chart would suggest. To increase advertisement awareness (and ultimately effectiveness), it was suggested to the client that they could spend money on some of the more expensive advertising channels (such as television or radio). However, the extra analysis also provided the conclusion that the client could capitalize on their competitors’ messaging by contrasting it with their own, potentially using their competitors’ advertisements to their advantage.

To recap, simply asking respondents questions to gauge their ad awareness and effectiveness will not provide much competitive intelligence, nor will it help identify acquiescence bias. Since asking the same questions for all competitors is not feasible, consider including a few competitors’ options as well as some fictitious options. Doing so will provide the missing context, help determine potential bias, and keep surveys simple for respondents.

Patrick Kyba

is a Consultant with Advanis. Patrick plays a lead research role on projects having responsibility for survey development, survey programming and testing, development of analysis plans, data cleaning, weighting, and report writing and production. He strives to find the best reporting solution for his clients, from simple tables and graphs to complex automated online reporting or detailed custom reports.