Journal Probe of Lab Test Results Sparks Furor
A highly unusual inquiry by a respected scientific journal has precipitated an international shouting match between a group of French researchers and the journal team that is investigating them.
In a display of public anger rarely seen in the scientific world, the researchers in question, headed by Jacques Benveniste of INSERM, the French equivalent of the U.S. National Institutes of Health, Tuesday called the inquiry by the British journal Nature “outrageous” and “a mockery of scientific inquiry.” He said the behavior of the investigators from the journal was “completely unprofessional. This whole thing stinks.”
Results of the inquiry, to be published in Thursday’s edition of Nature, suggest that controversial results obtained in the French allergy laboratory are the result of “shoddy science” and the researchers’ own “delusions” rather than confirmation of an unusual scientific phenomenon.
Bizarre ‘Memory’ Suggested
The genesis of the dispute was a paper by Benveniste, published in the June 30 edition of Nature, that purported to show a biological effect produced by solutions that were theoretically too dilute to cause such an effect. Taken at face value, the report suggested that the solutions had some form of bizarre “memory” of substances they once contained.
Benveniste’s report has attracted much attention because of both the unusual circumstances under which it was published and the credence it seemed to lend to the folk-medicine practice of homeopathy, in which exceptionally small amounts of toxic materials are often used therapeutically.
Benveniste’s results had been confirmed by five laboratories in four countries. Nonetheless, Nature--in a unique “editorial reservation”--termed them “unbelievable” and noted that they defied common sense. The paper was published only on condition that Nature could send a team of observers to view a repetition of the experiments.
The investigative team was sharply criticized by Benveniste, and even Nature conceded that the team was “odd.” The team members were Nature editor John Maddox, a journalist trained as a physicist; scientific fraud investigator Walter Stewart of the National Institutes of Health, and magician James (The Amazing) Randi, who specializes in exposing scientific and religious charlatans. The team did not contain anyone who specialized in immunology.
“We thought of that (the lack of an immunologist),” Maddox said in a telephone interview, “but it seemed quicker to do it this way. We were looking for very obvious flaws. We knew that if we got into the intricacies of the technique and the immune system, it would take much longer.”
But, Benveniste said in an interview, “sending a magician to my lab implied trickery from the start. I was very concerned about that . . . but I was under the constant pressure that if I refused anything, that meant I had something to hide.”
The team visited Benveniste’s lab during the first week of July, and both sides conceded that the atmosphere was tense and confrontational. They observed seven sets of experiments and studied notebooks containing data from previous experiments.
They concluded that “the design of the experiments . . . is inadequate as a basis for the claims made last month. . . . We believe that experimental data have been uncritically assessed and their imperfections inadequately reported. We believe that the laboratory has fostered and then cherished a delusion about the interpretation of the data.”
In particular, the team concluded that Benveniste’s results were not totally objective because technicians knew what data was expected from each sample. The technicians’ observations were thus unwittingly influenced--leading them to report values close to those predicted from theory, with little of the randomness expected in the results of any scientific experiment, according to the inquiry report.
‘Too Good to Be True’
“Their results were, in essence, simply too good to be true,” Stewart said. “Observer bias could account for everything we saw.” He noted that in the three “failed” experiments in which the samples’ identities were fully hidden, randomness of data was present.
“We cannot imply that there was any cheating going on,” Randi said. “They were simply rounding off their numbers a little bit and ignoring data that didn’t agree with their theory. They would rather throw out ‘bad’ data than throw out the theory.”
Benveniste, meanwhile, said Nature’s objections are based “on one dilution series on two patients’ blood taken when our technicians were tired and harassed. How can you let five years of work go down the drain based on one dilution series?” He also argued that the Nature critique misstates the results of investigators in other laboratories who have reproduced his work.
He said he will continue the work. “We have always said there was the possibility of an error of methodology, and that possibility is still open. I will certainly have these experiments done again.”
And he has a few words of advice for his colleagues: “Never, but never . . . let these people get in your lab. Scientists must not be treated like criminals.”
