Why Poll Results Differ
Earlier this week, Field Research Corp. released a poll on the election to recall Gov. Gray Davis. Today, The Times Poll comes out with somewhat different results.
The two polls are more alike than different. Both found the same lineup among the top candidates to replace Gov. Gray Davis -- Lt. Gov. Cruz Bustamante in the lead with support from 30% of likely voters surveyed, Arnold Schwarzenegger in second place with 25% and state Sen. Tom McClintock in third place.
And both surveys found that, on the first part of the ballot -- whether Davis should be removed from office -- the race had tightened somewhat.
But on two important numbers, the polls differ:
The Field Poll pegged McClintock’s support at 13%; The Times Poll put him at 18%.
More noticeably, the Field Poll showed the recall ahead 55%-40%, but The Times Poll found the race to be tighter, 50%-47%.
A number of factors may account for the differences between two surveys that have long track records of accurately gauging California elections.
Part of the difference may simply be the margin of error in the two surveys. Polls are based on statistical probabilities. In the case of the Field Poll, which is based on a survey of 505 likely voters statewide, the probability is that the reported results are within 4.5 percentage points either way of what one would expect to find if all likely voters in California were surveyed. Pollsters call that a 4.5-point margin of error.
In the case of The Times Poll, the margin of error is smaller, three percentage points, because the sample size is considerably larger -- 922 likely voters statewide.
Those margins overlap. When the Field Poll says the “yes” vote on the recall is 55%, that means pollsters are confident that the figure is between 50.5% and 59.5%. For The Times Poll, the corresponding range would be 47% to 53%.
But margin of error probably does not explain everything.
Other factors that can cause polls to differ run a broad range. Poll questions are worded differently, placed in different orders, juxtaposed in different ways. Each of those differences can cause slight variations in the results.
Then there is timing.
“Polling is a snapshot in time,” said Susan Pinkus, director of The Times Poll. The Field Poll was conducted from Sept. 3-7, while The Times Poll was conducted Sept. 6-10.
Voter sentiment almost always fluctuates over time, so polls taken on different days almost always have different results.
In addition, the people surveyed in the two polls differ in some important ways.
Of the likely voters in The Times Poll sample, 38% identified themselves as conservatives. In the Field Poll, only 32% identified themselves as conservatives. That difference could partially explain McClintock’s better showing in the Times survey.
In last November’s election for governor, 35% of those who voted identified themselves on exit polls as conservatives. If this year’s electorate resembles last year’s turnout, The Times Poll would have slightly too many conservatives in it, while the Field sample would have slightly too few.
In the last election, 35% of people identified themselves as liberals. In The Times Poll sample, 34% of those surveyed said they were liberal. But the Field Poll sample included only 25% liberals. (In the Field sample, 43% called themselves moderates, compared with 30% in last year’s election).
The smaller number of liberals in the Field Poll could partially explain why it had a lower figure than The Times Poll for the anti-recall vote.
Pollsters try to ensure that their samples accurately reflect what they think the voter turnout will be. But of course, no one knows for sure.
Turnout can affect an election in two ways -- the overall level of voters and how they break down by political leaning.
The Times Poll uses a series of questions to determine how likely a person is to vote. Then, depending on what level of overall turnout is predicted for the election, survey participants are ranked as likely voters.
To examine the effect of a high versus a low turnout, The Times Poll experimented with different approaches for determining how many voters to count as likely.
The results indicated that the overall level of turnout probably would not have much effect on the outcome of the recall election.
The one exception was that if turnout is extremely low -- with only the very likeliest voters going to the polls -- Davis would do somewhat better.
The makeup of the turnout -- which side does a better job of motivating its voters to go to the polls -- always has a big effect on outcomes.
That, of course, is what campaigns are about.