On the Media: Survey shows not all polls equal

Virtually every single American supports the Arizona crackdown on illegal aliens. Half of the country believes President Obama’s immigration policy exposes him as “the most divisive, partisan, anti-American president in history.”

These fun factoids come to us courtesy of the world of online insta-polls. Click in your vote. Press on results. And voila — since you’re likely visiting a site patronized by your ideological doppelgangers — your world-view reigns supreme!

These Internet surveys sometimes acknowledge how unscientific (read: meaningless) they really are. They surely must be a pale imitation of the rigorous, carefully sampled, thoroughly transparent polls favored by political savants and mainstream news organizations, right?

Well, er, sort of. The line between junk and credible polling remains. But it became a little blurrier — creating concern among professional survey organizations and reason for greater skepticism by all of us — because of charges this week that one widely cited pollster may have fabricated data or manipulated it so seriously as to render it meaningless.

Blogger Markos Moulitsas, founder of the left-leaning Daily Kos website, filed a lawsuit in federal court in Oakland on Wednesday charging that Research 2000, the organization he had commissioned for 1 1/2 years to test voter opinion, had doctored its results.

The firm’s protestations that it did nothing wrong have been loud and repeated. Evidence against the company is somewhat arcane. Suffice it to say that independent statisticians have found a bewildering lack of statistical “noise” in the company’s data. Where random variation would be expected, results are too consistent.

Moulitsas has said he became particularly alarmed after outside experts raised doubts about the firm’s methods and then the pollsters declined to supply basic information to confirm, among other things, that they had actually interviewed the people they said they had interviewed.

The Daily Kos founder said he hopes the litigation will reveal what work the firm actually completed. He plans next week to announce the hiring of a new polling firm to support the Daily Kos — which claims 250,000 registered users and 2.5 million unique monthly users.

“We’ve learned our lesson,” Moulitsas said in an interview. “We are going to be doing our due diligence this time. A lot more things have to be nailed down, like making sure we have access to the raw data.”

Research 2000 might be easily written off as ideologically tainted if it worked only for the persistently liberal, mostly pro- Democratic Party Daily Kos. But a bunch of long-time news organizations — multiple TV stations and papers such as the St. Louis Post-Dispatch and Reno Gazette-Journal — also trusted the Maryland-based outfit.

Said Moulitsas in a post this week: “I got burned and got burned bad.”

The revelations have professional pollsters in a tizzy of self-examination. “The implications are ominous,” wrote Mark Blumenthal, a columnist for National Journal Online and founder of poll aggregator Pollster.com. “The polling industry cannot simply continue on a business-as-usual course.”

But what to do?

Some questions about polling will never be settled definitively. Much of the profession’s work relies on science, but a measure of art also comes into play in trying to assess human beliefs and emotions. Each firm will come up with a unique model this fall, for example, to decide just how energized Republicans have become and, therefore, how many will turn out in the November mid-term election.

Most reputable pollsters agree on one thing — polling organizations should publicly disclose as much of their methodology as possible. Just for starters, they should reveal how many people were interviewed, how they were selected, how many rejected the survey, how “likely voters” and other sub-groups were defined and how the raw data was weighted to reflect the population, or subgroups.

In the meantime, we in the audience —- who don’t fully understand the influence of all those important choices — will continue to be bombarded by polls. Many of them will be the cut-rate, automated “robo” polls, in which a prerecorded telephone questioner might be begging information from a grade-schooler.

Michael Cornfield, a George Washington University political scientist and polling expert, recommends that concerned citizens ignore the lone, sometimes sensational, poll result. “Trend data are superior to a single point in time,” Cornfield said via e-mail, “and consensus results from multiple firms are superior to those conducted by a single outfit.”

Websites like RealClearPolitics.com and Pollster.com average results from many firms to try to get a more accurate picture of what the public really thinks. Pollster.com’s most recent average of surveys found, for example, that 48.9% disapprove of the Obama’s job performance, while 45.8% approve.

An easy-to-read chart also makes a longer-term trend clear: The president’s popularity, once solidly more than 60%, has been slipping fairly steadily since he took office in 2009.

Pollster.com also allows the curious to examine how individual pollsters have performed. Rasmussen Reports, for example, consistently finds a more negative reaction to Obama than other surveys, sometimes by a wide margin. Filtering out the findings of Fox News-friendly Rasmussen, the average of other polls show Obama’s approval/disapproval numbers almost evenly divided.

According to Nate Silver, a statistician who rates polls, New Jersey-based Rasmussen also stands apart from many other top polling operations in that it has not committed to industry standards for disclosing its methodology.

Those details are just so cumbersome, so heavy. And so heavily important for experts trying to determine why a survey comes out the way it does.

The rest of us should look at none of the polls. Or look at all of them. And look out for the operators not willing to tell us how they’re doing business.

james.rainey@latimes.com

Twitter: latimesrainey