How to Tell if Political Polls Are About Truth, or Consequences

When Harvard University’s “Vanishing Voter” project asked Americans this month who they supported for president, the winner was overwhelming.

Close to 75% said they supported “no candidate.”

George W. Bush was next with 13%, followed by Al Gore, 6%; John McCain, 3%; and Bill Bradley, 2%.

Those findings contradicted polls showing Bush with up to 56% of the vote. Why?

Because pollsters asked the questions differently.

The other surveys only give voters the names of candidates appearing on the ballot, said Thomas Patterson, co-director of the Vanishing Voter project. “We posed the question differently: ‘Which candidate do you support at this time, or haven’t you picked a candidate yet?’ ”

This doesn’t mean that polls showing Bush ahead nationally are wrong. The apparent contradiction shows that results can be influenced by many factors, such as the wording, or even the order, of questions.

The goals of polls also may differ. The aim of the Harvard poll, for example, is to determine why people don’t vote, not which candidate is leading the pack.

Public Opinion Can Change Daily

The good news is that there is a great deal of reliable information to be gained from reputable polls and surveys, experts say. There are even two industry-watchdog groups, the National Council on Public Polls and the American Assn. for Public Opinion Research, that set standards and censure rogue polling outfits if necessary.

With New Hampshire’s first-in-the-nation primary two days away, it’s good to remember that polls have limitations. Public opinion can change daily. “Polling doesn’t predict the future; it captures a snapshot of a moment in time,” said Susan Pinkus, director of The Times Poll.

Consider “margin of error,” which is determined through mathematical formulas. Say candidate Jones has 43% and Smith 37%, but the margin of error is plus or minus 4 percentage points. Jones’ lead may not exist.

Because the margin of error is both negative and positive, you have to double it. Therefore, Jones needs to lead by more than 8 percentage points to exceed the margin of error and be considered a clear front-runner.

If polled properly, statisticians say, as few as 1,000 people can accurately reflect opinions of 185 million Americans. The most widely accepted method of sampling people is through random-digit dialing, which means everyone has an equal chance of being interviewed.

Using area codes and exchanges in geographically stratified areas, computers generate more than 3,000 phone numbers to make up for hang-ups, refusals, those who aren’t home, non-residences and other dead-ends. Computer-generated dialing is vital, especially in California, which leads the nation in unlisted telephone numbers.

Weighted formulas are used to make sure men and women, minorities and age groups are represented in the same proportion in which they appear in the general population. For political surveys, pollsters ask questions regarding voting history and intention to vote to identify likely or highly probable voters.

Opinion Usually Set by the Weekend

As a key primary or election nears, “tracking” polls begin. Rather than taking a snapshot in time, as most surveys do, tracking polls use different methods, such as rolling averages, to detect daily opinion shifts. In other words, pollsters might average five days’ responses to get their results. Each day, the oldest polling numbers would be deleted in favor of the newest, which would track public opinion as it evolved.

Dick Bennett, president of American Research Group in Concord, N.H., tracks presidential campaigns in that state. In 1996, his firm was the only one that showed Pat Buchanan would defeat Bob Dole in the Republican primary.

Bennett says voters usually have made up their minds by the Saturday before an election. He can tell for a simple reason: They clam up when his interviewers call. Willing for months to share their opinions, voters become suddenly shy right before the election because they’ve committed themselves and now view their vote as a private matter. His surveyors work persistently to get these voters to talk, he says.

Others have mixed feelings about late polling. Linda Fowler, a Dartmouth College government professor who conducts joint polls with the Associated Press, says undecided last-minute voters can be unduly swayed by surveys.

“They tend to reinforce the front-runner status,” she said. She releases her last poll the Friday before an election.

Bennett agrees that New Hampshire’s independent voters, especially, who can register on primary day, look at polls carefully. But he thinks it’s information they deserve.

“That new voter is somewhat strategic. . . . They want their vote to count,” he said. “They want to go for a winner.”

There are many reputable polling outfits, some teamed with major news outlets, who do public opinion surveys. But there is a seamier side to polling and, technically, it’s not scientific polling at all.

For instance, a “push poll” will say to a possible voter: “If you found out that Candidate Jones had beaten his wife or defrauded a business partner, would it change your vote?” The aim is not so much to determine public opinion as to shape it by smearing an opponent.

“It’s not true,” said Pinkus of the question, “but it puts something negative in the voter’s mind.” Push-polling is no stranger to presidential politics. Examples of it popped up in Iowa before the caucuses.

Dial-in and mail-in polls also should be ignored, experts say, because people “self-select” to respond. Those who choose to participate don’t necessarily represent the general public. For instance, a 900-number survey might attract 70,000 people opposing gun control and 30,000 favoring it, which would contradict scientific surveys showing a majority of Americans in favor, Pinkus said. Pro-gun forces could organize phone trees to flood the phone lines, and some people could vote more than once.

One new kind of poll that is attracting lots of interest is Internet polling. But it troubles some politicians and pollsters.

For now, even scientifically conducted Internet polling is dicey because millions of Americans are still not online, with poor people and minorities underrepresented. But that is changing. Experts say legitimate online polling could prevail someday, just as phone samples replaced arduous door-to-door surveys 40 years ago.

(BEGIN TEXT OF INFOBOX / INFOGRAPHIC)

Polling the Public

Do you agree strongly, agree somewhat, disagree somewhat or disagree strongly with the following statement? “I understand how polling works and how it influences politics.” If you tend toward the negative replies, this guide might be useful.

BY THE NUMBERS

So why, you might ask, haven’t I ever been interviewed for a poll? That would take a lot of polls. Here’s how it breaks down mathematically, according to the National Council on Public Polls:

*

200 million: U.S. voting-age population

*

1,000: Average size of a poll

*

200,000: How many polls it would take to interview every voting-age American, assuming no one is called twice.

*

250: Average number of national and local media political polls conducted each year.

OTHER POLLS Push Polls

Used by partisan political pollsters for their clients who are running for office. They are intended to push voters away from one candidate and toward another.

*

Dial-in Polls

Unscientific and not true polling. The sample is distorted because the respondents have to call in.

*

“Frugging”

The practice by special-interest groups or political parties of sending out questionnaires on a particular topic, often coupled with a plea for money. The results, also unscientific, are sometimes presented as poll findings.

*

“Sugging”

The practice of companies’ sending out questionnaires and then trying to sell something to the respondents. Again, this is not a real poll.

*

Internet Polls

Though sometimes amusing to read and increasingly popular, these are not true polls because the respondents are not selected randomly. Some Internet polls are labeled “for entertainment only” and, at a minimum, should indicate they are unscientific.

WORDING IS EVERYTHINGDifferent questions can elicit different responses.

Consider these questions, all related to the same topic:

1. Do you favor or oppose affirmative action programs to help women and minorities get better jobs and education?

2. Do you favor or oppose affirmative action programs that use quotas to help women and minorities get jobs or education?

3. Do you favor or oppose programs to rectify discrimination in jobs and education against women and minorities?

*

Did your answer change with each question?