Report discounts value of many ‘biomarkers’ in predicting disease
To gauge a patient’s risk for disease, doctors often look at blood levels of certain proteins or at other “biomarkers” — cholesterol to gauge heart risk, say, or bone density for fracture risk. And every day, more biomarkers are found, and are often described in the news with much fanfare.
But a study published Tuesday casts serious doubt on the predictive value of many of them.
After reviewing 35 widely cited research reports linking a substance to a disease, the study’s authors found that about 85% of the time, the strength of those links didn’t hold up when larger, follow-up studies were done.
In fact, in some cases, the connection between the biomarker and disease risk disappeared altogether, they reported in the Journal of the American Medical Assn.
“We hear every day that there’s a new marker for heart disease, cancer and so forth,” said study coauthor John Ioannidis, a professor of medicine and chief of the Prevention Research Center at the Stanford University School of Medicine. “That needs to be seen with a more tempered perspective.”
Interest in discovering new biomarkers — genes, proteins in the blood, infectious agents and more — has skyrocketed as researchers have learned more about the human genome and have begun to delve into how our genetic material controls what happens in the body, molecule by molecule.
More than 100,000 papers have been published about biomarkers, Ioannidis said. The National Institutes of Health has invested billions of dollars on biomarker research. The hope is that discovering new biomarkers will help physicians diagnose diseases in their earliest stages, when they are easiest to treat, or even flag a risk so that doctors can intervene before a disease develops.
“Biomarkers are very useful in medicine,” said Nader Rifai, a professor of pathology at Harvard Medical School and editor of the journal Clinical Chemistry, which routinely publishes biomarker studies. “We are always on the lookout for more.”
But the report published Tuesday suggests this type of research is only in its infancy.
Take the example of a 1994 study showing that patients who had a particular mutation in the BRCA1 gene were four times more likely to develop colon cancer than people without the mutation.
That study, published in the Lancet, gave women who were already nervous about their heightened risk of breast and ovarian cancer one more thing to fear. But 11 years later, an analysis that pooled the results of many studies calculated that having the BRCA1 mutation increased the risk of colon cancer by only 19%, and the largest study in the batch showed that the mutation did not increase the risk of colon cancer at all. (The mutation is still believed to increase the risk of breast and ovarian cancer.)
Or consider a 1991 study reporting that patients who had evidence of Helicobacter pylori infection were 3.6 times more likely to develop gastric cancer than people without the bacterium. That report, published in the New England Journal of Medicine, was cited 2,458 times in subsequent journal articles.
But a later analysis of multiple studies found the risk was merely doubled, and the largest individual study found that H. pylori increased the risk of gastric cancer by only 31%.
Ioannidis, who has made a name for himself as a crusader against bias in medical studies, said several factors combined to exaggerate the importance of biomarkers.
A lot of the studies that first draw attention to the markers are small. Statistically speaking, smaller studies with fewer people are more prone to random findings, and larger studies are better equipped to uncover the underlying truth, he said.
Also, scientists are more likely to cite studies with dramatic, positive results than those with lukewarm findings.
Even when a relationship between a biomarker and a disease is clear, it isn’t necessarily useful, Ioannidis said. The effect of the biomarker has to be large enough to make a significant difference in disease risk, and when it is, there has to be a treatment available, he said.
“It’s a first step, but it’s not the end of the story,” he said.
Patrick Bossuyt, a clinical epidemiologist at the University of Amsterdam in the Netherlands who wrote an editorial accompanying the study, said its findings were not surprising.
“There are well-founded hopes for the discovery of biomarkers, but there are serious deficiencies in the way it’s being done,” said Bossuyt, who was not involved in the research. “I’m still very optimistic, but I think progress will be a lot slower than many people hope or believe.”
That a crusader against bias in medical studies would find bias in medical studies poses a conundrum: Is it possible that Ioannidis also is biased in favor of finding problems with medical research?
“I suspect there may be a risk, indeed, of being biased in favor of bias! I think I have to be careful about it,” he wrote in an email.
