Science by Headline: a Dangerous Practice

This month, the New England Journal of Medicine posted on its Web site an article trumpeting the apparently beneficial results found in a study of a new medication for breast cancer, letrozole.

There were two worrisome things about this. First, the study was published on the Web four weeks before being published in the journal. Second, the study itself was supposed to last five years but was abruptly ended after only 2 1/2.

Why couldn’t the journal wait a month? Why didn’t the researchers complete their study?

The answers may begin with the enormous financial and political pressures to get new drugs to market fast.

Drug testing takes considerable time and vast sums of money. Patients clamor for any drugs that might help them, the media are always on the hunt for news, and scientists are always seeking breakthroughs. So the temptation to cut a drug trial short and to overemphasize results that merely seem promising is often overwhelming for researchers and the drug companies that fund them. The patience to wait and do it right seems to be fading.

In this latest study of letrozole, known commercially as Femara, more than 5,000 post-menopausal breast cancer patients were studied to determine whether adding five years of treatment with letrozole improved the disease-free survival of those who had already received five years of treatment with Tamoxifen.

The survival rates of the letrozole group did not differ significantly from a control group that was given a placebo. However, the letrozole group was said to have a statistically significant 46% decrease in the risk of a recurrent or new breast cancer.

Does that sound impressive? It’s not. For one thing, the two groups were not matched by the extent of cancer at the time of their first surgery, nor were they matched by the type of chemotherapy they had. These differences -- and not letrozole -- might have affected the recurrence of breast cancer.

Second, the researchers were reporting “projected” results, not actual ones. Because the study was stopped prematurely, none of the women had actually received the full five years of letrozole.

Third, although 46% sounds like an impressive decrease in risk, it’s a statistical manipulation. The absolute decrease in risk was only 6%. The New England Journal’s own editorial acknowledged that even if the beneficial effect reported in this study were valid, the use of letrozole would prevent only one breast cancer occurrence for every 100 women treated.

Yet we already are reading letters to newspapers from people saying, “Thank God the researchers halted this study early so that we may benefit! If only my beloved sister (mother) (wife) (daughter) had had this amazing drug!” That is the reaction the hoopla is designed to generate, and that is what troubles us.

We are distressed by the decision of the investigators to terminate the letrozole study before they could get more definitive answers about the recurrence of the disease and about the women’s overall survival. And we are even more distressed by the New England Journal of Medicine’s decision to create an atmosphere of drama and urgency by its early release of the article.

More than 20 years ago, Allen L. Hammond of the American Assn. for the Advancement of Science cautioned the public that “in today’s news-conscious world, there is an enormous emphasis on breakthroughs. But with rare exceptions, science is a process, not an isolated event. Conveying the way science really works, the interplay of persistence and luck, the painstaking accumulation of evidence, the clash of proponent and critic, the gradual dawning of convictions demands a look behind the headlines.”

His observation is even more crucial in medicine, where scientific discoveries can have grave consequences for life and death. But how many times have the headlines blared news of some new miracle drug, followed -- as night does day -- by later news of the drug’s side effects, ineffectiveness or risks?

All of us, consumers and physicians, would do well to look behind the headlines of medical “breakthroughs” and remember that headlines sell news -- and news sells drugs.