2-Sided Ballot Doesn’t Fly in Canadian Study

A team of psychologists Thursday said it has come up with the first experimental evidence that the controversial “butterfly ballot” used in Palm Beach County, Fla., introduced systematic voting errors “which call into question the validity” of the presidential election results.

But the study itself is already proving controversial, with charges that it is too small to be taken seriously and that it was rushed into print for political reasons.

The experiments, conducted in Canada one day after the U.S. election, showed people who voted on a butterfly, or two-sided, ballot, were more confused than those voting on a single-column ballot, the researchers reported.

When 161 college students used the butterfly ballot, they reported being confused but did not make errors. But when 53 “real people” recruited at a shopping mall used the ballot, they made four errors--a rate of about 8%.

The test ballot contained Canadian names. Based on the location of those names, the team concluded that three of the errors were the equivalent of accidentally voting for the Reform Party’s Pat Buchanan instead of Democratic candidate Al Gore. Because three of the errors were identical, they concluded that the ballot caused systematic errors.

“The message is clear here. It doesn’t take a PhD to look at the ballot they used and say it was confusing,” said Robert C. Sinclair, a psychologist at the University of Alberta and lead author of the study.

The study was published on the Web site of the international science journal Nature. The results were released early, Nature editors said, because of their immediate interest.

The study adds to evidence that “overwhelmingly suggests the ballot had a problem,” said UC Berkeley political scientist Henry E. Brady.

In their own analyses of voting patterns, Brady and others suggest that voter error in Palm Beach County cost Gore as many as 9,500 votes--enough to win Florida and the presidency. Those studies indicate Buchanan, a conservative, obtained more votes in liberal precincts than in conservative ones.

Kenneth W. Shotts, a political scientist at Northwestern University in Chicago, said that, although he is convinced the butterfly ballot cost Gore the win, he remains unconvinced by the report published in Nature. “This study’s dramatic conclusion is based on extremely weak evidence.”

The study’s main flaw, he argued, is that it is too small. Because it focuses on analyzing errors--and the error rate in Palm Beach County was just slightly higher than 1%--he said the ballot should have been tested on more people. The study included only 15 people who intended to vote for Gore on the butterfly ballot.

Other critics are skeptical of the report’s claims about ballot problems because college students using it made no errors at all. But Sinclair said students performed better because they are skilled at using complicated testing forms. As for the small sample size, he said, “this is an experiment, not a survey.”

The results, he emphasized, meet a critical test. They are statistically significant, meaning they are likely to be real effects that did not occur by chance. Scientists are often skeptical of smaller studies, though, because they are more likely to allow events caused by chance to appear as patterns.

Others are suspicious of the study’s quick publication in Nature, a journal that routinely requires months of careful peer review and rewriting before reports are published.

“I’m somewhat astounded that a prestigious journal like Nature would publish this,” said Tim Groseclose, a conservative-leaning political economist at Stanford University’s business school and Hoover Institute.

Nature editors stood by their decision to publish the results early and said it was not an attempt to influence the election. “No one would claim this is a major contribution to science, but it is a scientific contribution to an issue that’s very topical,” said Laura Garwin, Nature’s North American editor.

Sinclair, a Canadian, is not affiliated with any American political party. An expert in survey design, he conducted his experiment out of intellectual interest. “I was up at 5 a.m. watching CNN. They flashed an image of the butterfly ballot on the screen, and I looked at it and said, ‘This is problematic.’ ”

In what amounts to lightning speed for a scientific investigation, he created a facsimile ballot, tested it on 324 undergraduates and analyzed the data in a single day.

Two days later, his team began testing the ballot in a shopping mall, where they could solicit volunteers more representative of the voting public.

The report is available at: https://www.nature.com/nature/

Other statistical analyses are available at: https://elections.fas.harvard.edu/