Tracking Public Opinion Is a Delicate Blend of Science, Art

So how do you really feel about abortion? Immigration, the economy or Iraq? Or the president? Or the governor?

With the fall election season heating up in California and other states, public opinion pollsters will be asking questions on such topics to thousands of people in a drive to pin down the ever-shifting mood of the country.

As Los Angeles Times Poll Director Susan Pinkus explains below, obtaining a valid result involves a keen blend of art and science.

Gallup, Roper, Field and newspapers such as The Times -- to name just a few major polling organizations -- begin with a bank of telephone callers who ask carefully written questions of people scientifically chosen at random. Randomness is a key that allows pollsters to accurately project the opinions of thousands based on a sample of only 1,000 or even hundreds.

The answers are entered into a computer and sorted against various statistical measures. The results are then analyzed in the aggregate and qualified with a mathematical “margin of error” that tells readers the results might vary within that range.

*

Question: What is the value of a public opinion poll?

Answer: It tells us what people are thinking at that particular moment about a particular issue. Polling is just a snapshot in time, and it’s not a predictor of the future. It’s just basically telling you, if there’s a [polling period] of let’s say, Oct. 1 to Oct. 7, that’s what voters were thinking about those issues during that period of time. We can get a sense of what Americans are feeling about the war in Iraq, for example, or the government response to Hurricane Katrina, or the special election in California.

*

Q: As a professional pollster, how do you react when you hear a politician say something like “Oh, I don’t believe in polls”?

A: I laugh. It’s probably because they have a result they don’t really like. Because I am sure that they all have their own pollsters, they all care about public opinion and they all want to be on the right side of it so that they get the votes the next time they run for election.

*

Q: Websites occasionally ask a “question of the day” and then show how people responded -- is that a poll?

A: It’s not a poll because it’s not a scientifically designed sample. You can have a website where 10,000 people get on it, saying whether you want John O’Hurley or Kelly Monaco to win “Dancing With the Stars.” That’s a simple issue and that’s a silly issue. But does it matter? No.

But it’s different when we’re talking about current events or timely issues, like [President] Bush’s job approval rating or whether you think abortion should be legal or not. When a website says “OK, click yes if you agree” there could be a systematic or a concerted effort from one major group to answer it multiple times. You don’t know if it’s children answering it or only adults answering it or whether they’ve answered it many times.

*

Q: So what makes a poll scientifically accurate?

A: We begin with a “probability sample,” which allows every adult 18 or over to be chosen to participate in the survey. We use a random-digit dialing technique that uses computer-generated telephone numbers.

And you only want to speak to one person in a household who is 18 or over. We don’t even let the interviewers decide whom to choose because for the most part women answer the phone, so if you didn’t balance it out you would have a very one-sided survey of more women answering than men. So even within the household we randomly pick somebody by asking to speak to the person with the most recent birthday.

It’s a scientifically produced sample, so that you get representative numbers of men and women. You compare some demographics to census data. A random sample will have the right amount of men and women, the right proportion of blacks, whites, Latinos and Asians.

*

Q: Does it matter what kinds of people are questioned?

A: It depends on what your poll is trying to do. If you’re getting close to an election, you want to look at “likely voters” because those are the people who are going to vote. You don’t want to look at everybody, because not everybody votes. We just saw that in our last presidential election: Only about 54% of the population voted.

*

Q: How do you determine who is a “likely voter”?

A: We ask the person we’re calling about their past voting history, their intention to vote, their interest in voting, are they a citizen, are they registered to vote, whether they are definitely or probably going to vote.

*

Q: Is there any magic to the way a poll question is phrased?

A: Polling is called an art and a science. The science is the sampling and the methodology, and the art is the way you design the survey questionnaire. You have to be very careful of the order in which you ask the questions so that one question doesn’t taint the next question.

And then you have to figure out a way of writing a question that hopefully is neutral. You don’t want to use any inflammatory language. Like for abortion you try not to use the words “pro life” or “pro choice” because that conjures up very different, very clear images. Affirmative action is another example; you don’t want to use the words “preferential treatment” or “quota” because those are loaded terms. You want to ask questions that people understand, that are not ambiguous; we try to write to a very understandable level; we use simple, clear language.

*

Q: How do you ask about initiatives?

A: Initiatives are very hard to do because people really don’t pay attention to them until the last couple of weeks leading up to the election. We first ask if they have heard anything about the proposition and mention what the title is and then say nothing else about it. And we get a very high “don’t know.”

Then what we say is “as you know, the proposition is ... “ and then what we do is read the ballot summary because this is what voters are going to see when they go into the voting booth. It doesn’t have any arguments for or against it at that particular point.

*

Q: What should people keep in mind in reading poll results?

A: They should look to see who sponsors the polls because some are sponsored by bipartisan groups, while others are conducted by independent polling organizations.

And they should look at the sample size and the dates of the interviewing period. They should look at the “margin of error” so they know if there’s some significance between answers or, in the case of elections, the yes and the no vote for each initiative or the vote between each candidate.

For example, if a poll shows 43% of respondents would vote for Politician X and 41% would vote for Politician Y, Politician X may not really be ahead if the margin of error is plus or minus 3 percentage points. If the distance between the candidates is closer than the margin of error, the result would be a statistical dead heat.

*

Q: Does a poll result, say in September, necessarily mean that will be the result when the election is held in November?

A: Absolutely not. If it’s a poll on initiatives, the campaign hasn’t started yet. And so once advertising comes out or people come out in support of it, one organization or another, public opinion changes and that’s what’s so wonderful about it. Because people start hearing about it, they start tuning in.

A poll in September tells you how the issue is doing at that moment. This is why the cliche “a snapshot in time” is entirely true.

*

(BEGIN TEXT OF INFOBOX)

Polls apart

Two polls got opposite results when they asked how Californians would vote on a spending initiative sought by Gov. Arnold Schwarzenegger. Each posed the question differently, however.

*

Public Policy Institute

of California

Background: Nonpartisan, polls on a variety of state issues.

How question was asked: Used language from the summary that voters will see on the ballot.

“Proposition 76 is called the ‘State Spending and School Funding Limits Initiative Constitutional Amendment.’ It limits state spending to the prior year’s level plus three years’ average revenue growth. It changes state minimum school funding requirements under Proposition 98. It permits the governor, under specified circumstances, to reduce budget appropriations of the governor’s choosing. State spending is likely to be reduced relative to current law, due to the additional spending limit and new power granted to the governor. Reductions could apply to schools and shift costs to other local governments. If the election were held today, would you vote yes or no on Proposition 76?”

* 28% -- yes

* 61% -- no

* 11% -- don’t know

*

McLaughlin & Associates

Background: Republican, polls for Schwarzenegger and conservative officials across the nation.

How question was asked: Included a summary question that used language “based upon the real messages and goals of each proposition.” *

“Gov. Arnold Schwarzenegger supports ballot measures which will be voted on in the November election. These ballot measures are meant to clean up the spending and budget mess in California, as well as clean up the political and special-interest influence in Sacramento. Specifically, Gov. Schwarzenegger supports measures that will make the state live within its means by spending only what it takes in, instead of always having budget deficits; reward good, quality teachers and make it easier to dismiss poor teachers; allow independent judges to draw congressional, state Senate and Assembly districts instead of the politicians, and allow voters to approve or reject those districts; and, last, give union rank-and-file members the right to give their permission before money can be taken out of their paychecks by union leaders and used to make political campaign contributions.

“Now, if you were voting on this ballot measure package today, would you vote yes in support of this ballot measure package, or would you vote no, against this package?”

* 48% -- yes

* 28% -- no

“Now, thinking about the individual measures in this package, how would you vote? Would you vote yes or no on each of the following? A measure that will control state spending to end state deficits and balance the state budget without raising taxes.”

* 66% -- yes

* 22% -- no

* As shown on Schwarzenegger’s campaign website, www.joinarnold.com

Sources: Public Policy Institute of California, California Recovery Team

*

Los Angeles Times