Do TV ads measure up? Survey says . . .
There’s an old saying that half of all money spent on advertising is wasted. The eternal, career-wrecking dilemma for advertisers is that nobody knows which half.
And yet, with research, there is hope. ACE Metrix, a Los Angeles consulting firm, this month unveiled its online advertising diagnostic, which it calls “the first on-demand TV advertising creative measurement service.”
ACE Metrix trots out a lot of analytical language to describe its service -- it all sounds respectably arcane and complicated -- but the bottom line is that the company tells companies whether their commercials are good enough, smart enough -- and whether, in fact, people like them. And in real time, too. The big yardstick the firm offers is something called the “advertising creative effectiveness” (ACE) score, which is a daringly holistic measure of an advertisement’s excellence. In other words, does the “creative” work?
Despite generations of focus-group testing and whole industries devoted to consumer response, nobody really has a handle on how effective the creative part of an ad is until it hits the airwaves, and even then it might take months to know whether the ad really worked, which is to say, did it increase sales or awareness. Conversely, if advertisers get a lot of instant feedback on their ads, the buzz is usually bad. Consider the recent failure of Microsoft’s first “Bing” spot.
“We’re not really scoring the creativity of the ad, per se,” says Steve Goldman, who with social scientist JuYoung Lee started the company three years ago. “We’re isolating the quality of the messaging.” And yet, he adds, “We’ve seen the ability of big ideas and great storytelling to move the needle.”
The firm’s subscription-based service ($100,000 a year and up) essentially diagnoses every new commercial that hits the airwaves, using 500 online respondents corralled by a third party. This is a vastly larger sampling than all but the most ambitious in-house agency focus groups. The survey-takers answer nine shrewdly worded questions that deal with the ads’ persuasiveness and “watchability.” These results, parsed by age and gender, are fed into the ACE Analyzer’s proprietary algorithms that produce scores of 1 to 1,000 in categories such as “desire,” “relevance,” “change,” “attention,” “information” and the all-important “likeability.”
For comparison’s sake, the system also generates a category segment average -- the average ACE score for all automotive or beer and wine advertisers, for example. Clients can then log on, usually within 24 hours of their ad airing, to see how well it was received and by whom. This instant feedback feature is one of ACE Metrix’s selling points.
“If you know what the response is early you can do something about it,” Goldman says. If in the data-mining advertisers realize their ad is working with a non-targeted demographic -- say, a fast-food commercial that resonates with older females as opposed to young males -- the advertiser can reallocate ad dollars, chasing the more receptive audience. This is how ACE Metrix helps clients know which half of advertising is working. It’s also a handy way to keep tabs on your competition’s advertising, the ratings for which you can follow on the ACE Analyzer. Goldman notes the through-the-roof response that the Hyundai buyers’ assurance ad received in January: “If you were a competitor, you could have taken action.”
Ad agencies can also pretest ads against those of market rivals (Coke vs. Pepsi, Honda vs. Toyota) while in the editing stage. ACE Metric’s database currently contains about 1,800 commercials.
At a minimum, the ACE scoring could provide some ad agency with much-needed cover in the event an ad goes horribly wrong.
“Our customers love being able to compare ads,” says Goldman, “because advertising is ultimately about competition.”
Personally, I couldn’t care less whether Hyundai sells more Genesis Coupes. However, I am kind of in love with the idea of juridical and God-like ACE scores, a seemingly objective measure of something that is inherently subjective. Is that even possible? Could we run Shakespeare and Marlowe through the same statistical cotton gin to find out who was better?
“There are a lot of metrics that measure what was once thought to be immeasurable,” staff scientist Lee says. “IQ, for instance, and SAT scores and credit risk.”
To gauge the elusiveness of all this, consider two relatively well-known ads for import beers. One is the Heineken ad called “Men Scream in Cooler” (Wieden & Kennedy), in which a group of friends visit a couple’s home to see their recent renovations. While their women are oohing and aahing over the wife’s shoe closet, the men are in another part of the house, screaming, crying and hugging each other with delight over the man’s walk-in “beer” closet, stocked with ice-cold Heineken.
The other ad is a Dos Equis spot called “A Man’s Reputation Expands Quickly” (Euro RSCG Worldwide), featuring the “most interesting man in the world” character.
Both slick, both perfectly produced, both with a deft, daft sense of humor. From inside the fishbowls of their ad agencies, they probably both looked good. Which one is “better”?
According to the ACE scores, the Heineken ad absolutely trounces the Dos Equis ad, 565 to 310; meanwhile, in the key demographic, males ages 21 to 35, the Heineken ad scores a stunning 703, compared with Dos Equis at 583.
That sounds about right.
What does it all mean? “It means the folks working for Dos Equis have got some work to do,” says Goldman.
--
