Truth by Consensus : Deciding which scientific findings to trust is an imprecise process that goes on in the lab as well as in court. : Science File / An exploration of issues and trends affecting science, medicine and the environment
It’s one thing for a scientist to have a brilliant idea, or conduct a brilliant experiment. It’s quite another for that idea or result to be accepted as scientific truth.
Ultimately, Nature will have her say, and further experiments will prove the scientist right or wrong. But while waiting for Nature’s verdict, scientists--like lawyers--have to persuade a jury of peers to accept their results.
A curious case of the sometimes shifting sands of scientific truth involves an elementary particle known as the top quark. Back in 1985, its discovery at a European laboratory was proclaimed on the front pages of papers around the world. A U.S. physicist, reading of the discovery, remembers being highly amused.
After all, he said, the detector that found the quark had been shut down months ago, and since that time, scientists had been arguing about how to interpret the results. The announcement, he said, “only means that the physicists finally agreed on what they saw.” They didn’t agree for long, and soon afterward, the “discovery” was retracted. Almost 10 years later, the quark was discovered again at Fermi National Laboratory near Chicago.
This time, the discovery was on much firmer ground. Even so, the top quark was not officially unveiled until after months of negotiation to get about 900 physicists to sign off on the final wording of the papers.
Physicists can take their time about such things. But scientific evidence in legal cases has to be considered valid before it is brought into the courtroom.
How do courts decide what evidence is scientific enough to be presented to a jury?
Until last year, the standards were relatively simple. The primary litmus test for scientific legitimacy was publication in a respectable scientific journal. In order to be published, the work must be reviewed by a group of fellow scientists, or peers.
Then, in a case against Merrill Dow Pharmaceuticals by a family who said their child had been harmed by the morning sickness drug Bendectin, the Supreme Court struck down the peer review standard.
The family wanted to bring into court scientific data that had not been published in peer-reviewed journals. Although the family lost the case, the court decided that peer review was far from a foolproof method for evaluating good science.
In fact, peer review is rather notorious for making mistakes, especially when it comes to cutting-edge science. Science historian Gerald Holton of Harvard University likes to point out that Albert Einstein’s seminal paper on relativity almost did not get published because there was only one person in all of Ger many who would believe it.
“A lot of correct and even great science did not pass peer review,” Holton said.
(In place of peer review, the Supreme Court passed the responsibility for evaluating the scientific “truth” back to the judges, who are now charged with evaluating a scientist’s credentials and methodology themselves.)
Deciding what science to trust, and what to throw out as shoddy, is something that goes on in the laboratory as well as in court.
A researcher’s peers may look as hard at the reliability and reputation of the experimenter as they do at the experimental results. Trusting the expert judgment of other researchers, says UCLA science historian Ted Porter, is both necessary and common in science. “It’s what people commonly call expertise,” he said.
*
Judgment played a central role recently in one of the most dramatic mathematical announcements of the century: the news that mathematician Andrew Wiles of Princeton University had finally solved the notorious problem known for 300 years as Fermat’s Last Theorem. Since Wiles’ proof was highly technical--and ran about 200 pages--very few mathematicians were qualified to serve as peer reviewers.
“In the past, in principle, you could always go and check out a proof for yourself,” said mathematician Keith Devlin, who is dean of science at St. Mary’s College in Moraga, Calif.
“But the complexity and amount of information [in Wiles’ proof, but also in mathematics in general] is now such that we have to rely on other people.”
Technically, Wiles’ proof has yet to receive an official stamp of “truth” from the mathematical community, although most mathematicians are convinced that he did, indeed, solve the ancient problem.
Confidence in the top quark results also relied on trust to some extent. Physicists believed them in part because they trusted the logic of the experiments and the reputations of the experimenters. The more complex and specialized the work, the more non-specialists are forced to rely on the expertise of the people doing it.
“Ideally, physicists establish truth by experiment, and . . . mathematicians establish truth by rigorous proof,” said Devlin. “But in truth, we’re at everybody else’s mercy.”
*
Neither legal experts nor scientists are clear about the ability of judges to decide what’s good science and what’s not. “The legal system doesn’t seem to be set up to deal with [scientific] thinking,” said Steven Austad, a biologist at the University of Idaho. “What we would really would like to know is relative probability of truth in all [kinds of evidence].”
Yet even if law used perfect scientific reasoning, it still might not find the ultimate truth--because not even scientists claim that there is one right answer for every complex question.
“The [legal] briefs seem to assume that if judges simply adopted ‘scientific principles’ in . . . evaluating expert testimony, the ‘correct’ result would become apparent,” said Joan Bertin, attorney and director of the Program on Gender, Science and Law at Columbia University.
“But the very application of accepted scientific principles often leads scientists themselves in different directions.”
What to do? “One does the best one can,” said science historian Lorraine Daston of the University of Chicago.
