Audience tests: Plot thickens

Columbia Pictures chief Harry Cohn had his own way of determining whether a movie played well. Sitting in the screening room, he’d use his rump as his guide. “This is a very good picture,” the mogul once said, “but it is exactly 19 minutes too long. Exactly 19 minutes ago, my [rump] started to itch, and right there, I know the audience would feel the same thing.”

Such was the mind-set that market research gurus Joe Farrell and Catherine Paura encountered when they headed to Hollywood in 1977. The task at hand: to bring the gut-oriented movie world up to the rest of American business when it came to product testing. Though data aren’t the whole picture, they argued, only a fool ignores the audience. Conversant with the players and adept at handholding, they became an institution in a numbers-resistant town. He was a born salesman and she had strong organizational skills, but they eventually became interchangeable.

Through the years, the pair developed a virtual lock on the $100 million that studios spend annually to test their movies and costly promotional campaigns. Their National Research Group led movie executives down a path where they, like politicians, have faced criticism for being “survey-driven” -- changing story lines, endings and pacing in response to findings. NRG data became a touchstone for studios laying out $90 million to produce and market an average film.

Having presided over this sea change, however, Farrell and Paura recently left the field (they’re making movies for Disney). Their departure -- along with increasing use of online data -- represents a watershed for Hollywood. It’s already changed the way some studios market and test films and created a free-for-all.

“It’s the wild, wild West -- and everyone is trying to take the reins,” one studio marketing chief observes.

While NRG (rebranded Nielsen NRG) remains the industry’s gold standard, it’s vying with challengers such as perennial No. 2 MarketCast, the computer-oriented OTX (Online Testing Exchange) and C.A. Walker, which are eager to fill the void.

Shaking things up even more are changes on the technological front. Online research is replacing personal and telephone contacts in key aspects of the business.

“Change is scary,” says Shelley Zalis, founder of OTX. “But I wanted to give companies that had been doing things the same way for 25 years tools that dig deeper into the moviegoer mind-set. Early on, I told the studios not to put all their eggs in one basket, because I may not come through. But every time there’s a difference between traditional pencil and paper surveys and online predictions, online does better.”

The new methodology is no substitute for recruited screenings, at which producers and directors “smell” an audience -- taking in the laughs, the boos, the applause. Nor are post-screening focus groups, at which people deliver their verdict on a movie, likely to be phased out. But when it comes to evaluating promotional materials, proponents say, the online approach is faster and cheaper than interviewing people at malls. And it’s more efficient than telephone surveys when assessing the awareness of -- and interest in -- a movie.

Dan Rosen, senior vice president of research at Warner Bros., says that even before the departure of Farrell and Paura, he was exploring alternatives because “mall intercepts” and telephone polling became less viable in the 1990s. NRG, wedded to those techniques, he maintains, was synonymous with the status quo.

“Anyone with an extra half-hour didn’t represent the sample you were going after -- the person in a hurry,” he says of the shoppers interviewed. “And people became more concerned with privacy, putting their names on ‘no call’ lists. .... We’re finally tapping into the Internet, like the rest of the research world.”

Paura compares the change in market research to the mid-1970s, when telephone research supplanted the door-to-door variety because of the rising crime rate and women entering the work force.

Kevin Yoder, who with his partner Howard Ballon runs NRG Nielsen, compares it to the mid-1940s, when the studio system began to erode.

“And the introduction of computers is similar to the introduction of TV, which the studios feared -- but ultimately embraced.”

An inevitable shift

Zalis, a self-described “computerphobe,” was one of the first to realize the limits of traditional methodology and the potential of the Internet approach for movie research. In 1998, she helped develop online research for A.C. Nielsen, but, convinced that corporate culture and technological innovation didn’t mesh, she created OTX -- an independent entity within IFILM Corp. 2 1/2 years ago. Warner Bros. and Sony were early supporters when only about a third of all U.S. homes had Internet access. Now that it’s double that, she works with most of the studios.

With online research, she maintains, you can interview more people for the same price and get a more representative sample. According to last year’s Simmons Research Consumer Study, 84% of all moviegoers who’ve seen at least one film in the last three months are now online users. Though wealthy, older men used to be disproportionately represented, gender, ethnic and income breakdown is now equivalent to that of the general moviegoing audience.

Though still in its infancy, online research has already revolutionized the field, says C.A. Walker President Sam Weinstein.

“Until now, there weren’t many ways to skin this thing,” he says. “The shift to the Internet is the biggest I’ve seen in my 20 years in the business. And without its stars, NRG is a little more mortal than it was.”

NRG’s Yoder and Ballon don’t take anything for granted. Having the largest data base and history with the studios gives them a leg up. But now that the founders have exited the scene, they know it’s a new horse race.

“We have to earn the business, building our core methodologies and bringing new tools into play,” says Yoder.

The old tools became a staple for anxious studio executives in search of a safety net. NRG screenings assessed a movie’s “playability,” and its focus groups evaluated key elements, such as humor and endings. In one famous example, NRG told Paramount Pictures that, rather than a suicide at the end of “Fatal Attraction,” people wanted retribution. Even though it didn’t conform to director Adrian Lyne’s vision, the studio reshot the last eight minutes so the mistress was shot by the wife. More recently, recruited audiences for the Adam Sandler comedy “Big Daddy” reacted poorly to the saccharine ending. So the filmmakers tacked on a party at “Hooters” to make it more fun.

“Both movies were big hits,” recalls Sid Ganis, a “Big Daddy” producer and former marketing chief at Paramount. “And without the research, we would only have been guessing.”

The online debate

Three months before the release of a film, print ads, trailers and TV commercials (the biggest slice of the marketing budget) were evaluated by intercepting people at malls, usually on weekends, when traffic is at its peak. Based on the results, studios could determine whether the message was getting through to the desired demographic. Then, just before a film’s release, telephone surveys (“tracking studies”) would measure how aware people were of a given movie and whether they intended to see it.

Online research opened a world of possibilities for the studios, permitting more focused polling.

“We can access everyone from Asian laxative users to African American women who saw a film last weekend,” says OTX President Kevin Goetz, who for 16 years conducted screenings and focus groups for NRG. “And unlike most mall intercepts, the data can be collected midweek and delivered the following day.”

Online methodology also reduces the possibility of interviewer fraud and bias, and variations in the way questions are posed and people are recruited. It’s also better able to reach the tough-to-penetrate youth market; OTX, for example, created an interactive character on a skateboard to attract teen respondents.

NRG has its own online wing: Reel Research, a hybrid that provides both Internet- and location-based research, overseen by Nielsen Entertainment. Relying solely on the former has its drawbacks, Farrell maintains. People without broadband capacity or high-speed DSL lines may lack the patience to download the requisite materials. Teens may be distracted while taking the survey -- listening to music, e-mailing friends, doing their homework. Plus, he asks, what’s to prevent a lonely 50-year-old woman from saying she’s an 18-year-old blond?

“I’m not saying computers aren’t the wave of the future -- we may be wearing them on our wrists soon,” Farrell says. “But one-on-one is still a surer, if more expensive, way to get things done. If research scatters, if it’s commissioned to many suppliers with different methodologies, it could break up the standards of information and quality that we worked so hard to establish.”

Not everyone was impressed by the traditional approach, however. Some claimed that Farrell and Paura manipulated the data and filtered out bad news. Others pointed to a conflict of interest because one company represented all of the studios. Addressing the criticism, studios say they hedged their bets. The NRG “green books” compiled on each movie were used only as tools, they insist, not as final determinants of strategy.

“We were aware they might be doing a bit of a sell job and had frustrations with what we knew was an inexact science,” Ganis said. “But we’d just modify the findings accordingly.”

Market data were also used by the studios as a way of predicting the all-important opening-weekend box office. These numbers were far from solid, particularly for films aimed at younger children and teens (this summer, for example, “Spy Kids 3-D: Game Over” did far better than predicted, while “Charlie’s Angels: Full Throttle” did far worse). And the forecasts became weapons used not only to hype a product but also to deflate the competition.

“Tracking studies, meant to provide strategic direction, have been twisted into a parlor game in which opening-weekend grosses are handicapped -- to the dollar,” says marketing consultant Perry Katz, former marketing chief for Universal Pictures. “It reminds me of an old Asian proverb: ‘He who gazes into a crystal ball often eats ground glass.’ ”

Market research was never intended to forecast the future, Paura says in defense of the methodology. NRG entered that field “by default.” Entertainment clients wanted the same kind of turnout prognoses as those delivered in the political arena -- and high-profile misses made everyone look bad.

Accessing teens online rather than trying to call them at home is easier and should enhance the accuracy of predictions for movies aimed at that demographic, says Zalis, who hopes to embark on a wireless, cellphone approach to box-office forecasts down the road. But Tom Sherak, a partner at Revolution Studios and former head of marketing at 20th Century Fox, says that if Hollywood lags behind the rest of the world in market research, the fault lies in the product. Each movie is individual, he notes, and you have less than a month to launch it. You can’t fine-tune it like a can of soup, because of its short shelf life. And you can’t adjust the price.

“You spend $90 million to make and market a movie, and it’s gone in two to three weeks,” he says. “We’re talking about big money. No wonder companies operating under the radar wasted no time moving in.”

To NRG’s Ballon, “it’s not about being afraid of the competition but being challenged by it. The status quo no longer works. Still, this brave new world will be defined less by methodology than by who delivers the goods.”