The article about C-reactive protein (CRP) in the November 14, 2002 issue of The New England Journal of Medicine (NEJM) made a great news story. A little known and inexpensive blood test that measures the level of inflammation in the body was found to better predict the risk of developing cardiovascular disease than the well-known cholesterol level.
The study found that among 28,000 women followed for eight years, the 20 percent with the highest CRP levels were 2.3 times more likely to develop cardiovascular disease than those in the lowest quintile. And much of this risk occurred independently of (and therefore would not have been identified by measuring only) cholesterol levels. The article concludes that identifying people with elevated CRP levels would allow “optimal targeting of statin therapy,” and that these people’s risk of developing cardiovascular disease may be decreased by the same statin drugs that are currently used to lower cholesterol.
This news provided an archetypal journalistic narrative: A widespread and previously unrecognized risk of serious disease can be identified by a breakthrough in medical research and treated with drugs already widely used to lower cholesterol.
My non-random sample of three newspapers—The Boston Globe, The New York Times, and The Washington Post—and two news magazines, Time and Newsweek—each carried at least one story based on this NEJM article. Collectively, the stories expressed great enthusiasm about the potential health benefit of the new test: “groundbreaking,” “the most promising advance in a long time,” “paradigm-shaking,” “extremely important,” “a home run.” In turn, coverage of this story certainly had great impact on increasing public understanding of the potential importance of measuring CRP levels routinely. Among those who read this news, it is likely that few want to miss out on the potential benefit of this medical breakthrough.
So what’s not to like? A closer look at the article in the NEJM and the news coverage that followed illustrates the terribly difficult challenges facing medical journalism today. Eight principles emerge that can and should be applied to reporting of research findings.
Principle 1: When only relative risks are reported, question the importance of the findings. The NEJM article reported that for women in the highest quintile of CRP level, the risk of developing cardiovascular disease was 2.3 times greater than it was for women in the lowest quintile. On the surface this sounds like a dramatic increase in the risk of serious disease. But how much does that really increase an individual’s risk? Nowhere in the NEJM article was the amount of increase in absolute risk reported. An answer can, however, be reconstructed from one of the graphs included in the article.
About one episode of cardiovascular disease developed among each 1,000 women in the lowest quintile of CRP levels each year. So the women in the highest CRP-level group would have had an absolute risk of approximately 2.3 episodes of cardiovascular disease per thousand women per year. This means that among 1,000 women with the highest quintile of CRP levels there were only 1.3 more episodes of cardiovascular disease each year than there were among 1,000 women with the lowest CRP levels.
The population used in this study was at such low risk of heart disease that the well-publicized reduction in relative risk was, in fact, of very limited clinical importance. Yet the data were presented in a way that precluded all but the most fastidious and statistically savvy of reporters and doctors from understanding the minimal importance of this seemingly dramatic increase in relative risk of disease.
This serves as a reminder that the clinical importance of reduction of relative risk cannot be evaluated and should not be reported without being accompanied by the absolute reduction in risk. But the news stories covering the NEJM article reported only the relative risk associated with an elevated CRP level. (The Washington Post mentioned that “in absolute terms” the risk of heart attack was “very small.”) Unfortunately, this is not the exception but the rule in medical journalism: Eighty-three percent of news stories that report the quantitative benefit of a new product give only the relative benefit.
Principle 2: When financial ties exist between researchers and the medical industry, the results of a study are 3.6 times more likely to be pro-industry. Be suspicious.
No corporate funding for the NEJM article was noted. The lead author did, however, disclose that he is “named as a co-inventor on patents filed by Brigham and Women’s Hospital that relate to the use of inflammatory biologic markers in cardiovascular disease.” Relationships between researchers and the medical industry are not unusual. Seventy percent of clinical research is now funded by the manufacturers of the drugs and medical products being studied.
Notwithstanding their excellent public relations work, corporations undertake this research first and foremost as a business activity. Their primary responsibility is to investors, not to the public’s health or financial well-being. The commercial consequences of research published in prestigious medical journals and publicized by the press can be enormous. With many millions, even billions of dollars at stake, the drug and device industries hire the best and the brightest researchers, writers, business people, and public relations firms to spin research findings to their advantage.
In the case of the NEJM article about CRP, there was selective reporting of the results. (The relative risk of cardiovascular events was clearly reported to be higher in the women with higher CRP levels, but the overall risk of death, which is even more important than cardiovascular risk, was not reported.) And there was a notable absence of adjustment for exercise and diet, which might have diminished the predictive value of the CRP level.
One might think that it would be easy for a reporter or doctor to query the data that had been used in the NEJM article to answer such questions. And one might also reasonably assume that an essential component of good reporting—not to mention of protecting the integrity of medical research— is transparency. But with commercially sponsored medical research, there is not even a pretense of transparency. Research data with commercial value are not available to the public, and often the complete data are not even available to researchers who participate in writing the paper.
The commercial bias does not stop with the research, but affects the way the results are reported to the public as well. In an interview for this piece (but not specifically in reference to the CRP article), Dr. Marcia Angell, former editor of the NEJM, told me that: “The authors of studies have every interest in presenting the results in the most spectacular possible way. The journals tend to go along with it. Medical research has gotten to be very much directed toward the media, just like everything else. Researchers want to get their studies covered by the media, which is sometimes parlayed into money for their institutions, as well as publicity for themselves.”
The techniques used to spin results can be very complicated and difficult if not impossible for most reporters to decipher without expert statistical assistance. These techniques are exquisitely catalogued in a paper by Lisa Bero and Drummond Rennie, deputy editor of the Journal of the American Medical Association (JAMA), entitled “Influences on the quality of published drug studies,” which was published in 1996 by the International Journal of Technology Assessment in Health Care. This article should be mandatory reading and a frequently used reference source for medical journalists.
Principle 3: When no conflicts of interest are declared, there might still be conflicts of interest that should be reported. Beyond funding of the study being reported, researchers are asked by medical journals to report financial connections that might bias their work. Few researchers believe their objectivity is compromised by their financial relationships with drug companies and other medically related industries. “A conflict of interest is being defined as something that biases your work, and you get to decide whether it does,” Angell explained to me. “That leaves a hole big enough to drive a truck through.”
In the CRP case, two of the researchers had coauthored an article in 2001 that had been funded by the manufacturer of the statin drug Pravachol, Bristol-Myers Squibb. And only four days after the NEJM article was published, a major new study was announced to evaluate whether the drug manufacturer AstraZeneca’s not-yet-FDA-approved cholesterol lowering drug will reduce the risk of heart disease in people with elevated CRP and normal cholesterol levels. The lead author of the NEJM article will be in charge of this new study. The commercial synergy between encouraging widespread adoption of CRP testing and increasing the market for cholesterol-lowering drugs is obvious. This highlights the importance of inquiring directly about any conflicts of interest that might be relevant to a story such as this.
Principle 4: Read the accompanying editorial, if there is one. It might do a lot of your work for you. In the CRP case, this would have been beneficial since the editorial accompanying the NEJM article was far less enthusiastic than the article. “Any clinical significance of the added value of C-reactive protein over conventional markers of coronary heart disease is debatable,” the editorial said. Yet its less sensationalized view of the research findings was mentioned in only two of the five stories about CRP (The Boston Globe and The New York Times).
Principle 5: Financial ties of all experts quoted should be included in the story. Five of the six news reports about the CRP study quoted the lead author’s enthusiastic endorsement of this “breakthrough” discovery: “a huge paradigm shift in how we think about cardiovascular disease;” “very powerful and I would even argue an overwhelming demonstration of the fact that it’s time to move beyond cholesterol;” “overwhelming evidence that inflammation is at least as important as LDL cholesterol,” and “continued reliance on LDL alone is not really serving our purpose very well.”
Only one of the five news stories (one of the two in The Boston Globe) mentioned that this researcher has any financial interest in advocating the widespread use of CRP testing or increased use of the statin drugs. Other experts, not involved with this study, also gave enthusiastic endorsement to the use of the new test, but no documentation of financial relationship that might influence their opinions was included in any of the stories. (The four newspaper stories also quoted experts who were less enthusiastic about widespread use of the CRP test; the magazines did not.)
When medical experts are quoted in news stories, the public reasonably assumes that their comments are guided solely by a desire to serve the public interest, reflect the integrity of their position and/or academic rank, and are independent of commercial relationships. But the intertwined nature of business and medical relationships is so pervasive today, and so poorly understood, that the public cannot reasonably evaluate expert comments without knowledge of the presence or absence of commercial ties. No experts should be quoted without clearly indicating, on the record, whether or not they have commercial ties to any companies that might benefit or be hurt by the issue at hand.
Principle 6: Place the research findings in the context of other research, especially about lifestyle changes. The women in the lowest CRP quintile had 57 percent less risk of developing cardiovascular disease than women in the highest quintile. A study published in the NEJM in 2000, with a population similar to the CRP study, showed that female nurses who exercise regularly, eat a healthy diet, maintain a normal body weight, do not smoke, and drink alcohol moderately have 83 percent less risk of developing heart disease than women who don’t do those things. And more than four out of five episodes of heart disease that developed in this study were due to lack of adherence to this healthy lifestyle.
Now that’s news.
Other studies have shown that simply eating fish once a week reduces the risk of heart disease by as much as statins or even moderate exercise reduces the mortality rate by at least as much as statin drug treatment of people with high cholesterol. But there is no commercial push to remind people of the dramatic benefits—far greater than statins—of a healthy lifestyle.
Principle 7: Check previous research papers by the same authors. Unreported conflicts of interest and other issues that might be relevant to the current story often appear in other scientific papers by the same authors. In the CRP case, two of the authors published a paper in the American Journal of Cardiology based on the same data set only seven months before the NEJM paper. In the earlier paper, cigarette smoking was noted to be responsible for about half of the cardiovascular disease in middle-aged women, and the study found that CRP levels correlate significantly with the total number of cigarettes ever smoked—for both current and former smokers. The elevation of CRP found in former smokers was almost as great as that seen in current smokers. So any study, especially one involving middle-aged women that is undertaken to determine the power of CRP level to predict cardiovascular risk—independent of other obvious risk factors—would have to take into account whether a person is a current, former or non-smoker.
Even though these three categories for smoking status were available in the data used for the current NEJM article about CRP (according to yet another paper written by some of these authors and published in the NEJM in 2000), the category “former smoker” was not included in the most recent NEJM article. Women were categorized simply as being current smokers or non-smokers. This might seem academic, but the small absolute increase in the risk of cardiovascular disease seen in the women with the highest levels of CRP could simply be a result of them being former smokers. If this were the case, adjusting the data for former smoking might significantly diminish the additional predictive power of CRP level.
Principle 8: Do the arithmetic on cost. A previous study coauthored by two of the authors of the current NEJM study on CRP showed that treatment with a statin drug, Pravachol (40 mg. per day) successfully lowers CRP levels. Assuming treatment of women in the highest quintile of CRP with a statin drug reduces their risk of cardiovascular disease by 40 percent (a generous assumption extrapolated from previous studies), and applying this percentage to the absolute risk above, then treatment with Pravachol would prevent fewer than one cardiovascular event per year in 1,000 women.
The yearly cost of treatment with this dose of Pravachol is $1,572 per person (at the Northeast chain of CVS pharmacies). So the cost of each cardiovascular event prevented by treating women with the highest CRP levels with Pravachol would be more than $1.7 million dollars in drugs alone, not including the lab tests and doctor visits necessary to monitor for adverse drug effects. This could be reduced to “only” about one million dollars if a lower dose of a less expensive statin drug was found to be effective.
Why Good Journalism Matters
The CRP article in NEJM got wide press coverage, but a far more important article that appeared in the JAMA one month later received almost none. The “Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial—Lipid Lowering Trial” (ALLHAT-LLT) was designed to determine whether more widespread use of statin drugs would be beneficial for people at high risk of developing cardiovascular disease. In this study, 10,000 people at high risk of developing cardiovascular disease were randomized either to be treated with a statin drug or continue with their “usual care” (about 20 percent of this population took statins) and followed for about five years.
The results were quite surprising: Increased use of statins in this high-risk population has no significant benefit in reducing the risk of heart disease or death. In fact, the greater use of statins in the subgroup of people with normal LDL (bad) cholesterol levels, fewer than 130, was associated with an 18 percent increase in risk of death from all causes—not statistically significant, but very suggestive. Yet none of the publications in my sample reported this very important and unexpected but commercially disadvantageous story.
What might seem initially like picayune and pedantic concerns about the CRP article and the resulting news coverage have enormous medical and economic consequences. The lead author of the NEJM article told The New York Times: “From 25 to 30 million healthy, middle-aged Americans are at far higher risk than they and their doctors understand them to be, because we’re not taking inflammatory factors into account.” His comment suggests that many of these people might benefit from statin drugs, in addition to lifestyle changes. This comes on top of the 2001 guidelines for the evaluation and treatment of elevated cholesterol levels, recommending the number of Americans taking statin drugs increase from 13 to 33 million.
The additional cost of putting another 45 or 50 million Americans on statins would be between $45 and $75 billion per year for the drugs alone, not including the cost of screening all adults with CRP blood tests and the additional doctor visits and blood tests needed to monitor treatment with statins. This would increase total drug costs in the United States by 30 to 50 percent. Commercially backed justifications for wider use of these drugs saturate the paid media as well as news stories. At the same time, journalists place much less emphasis on major research like the ALLHAT-LLT study that suggests statins might not only be unhelpful in people for whom the national guidelines recommend drug therapy, but might actually be harmful for high-risk people with normal cholesterol levels. And journalists often fail to mention the evidence that healthy lifestyle changes appear to be far more effective than statins at preventing cardiovascular disease as well as decreasing the risk of breast and colon cancer and increasing longevity.
Our health care system is on the brink of collapse because of problems stemming from runaway costs. In the midst of this quickening crisis, medical journalism faces enormous structural challenges. Most clinical research is fundamentally commercial activity—albeit cloaked in the guise of public service—designed to maximize corporate profits. While publishers depend on advertising revenues from the drug companies and other medical industries, readers yearn for stories of scientific progress that can keep chronic and acute illnesses at bay.
Impediments to Good Reporting
Unraveling the commercial bias that colors much of our medical research is a complicated undertaking, at best. Journalists often don’t have the statistical and research expertise to cut through the pro-industry spin, nor are they given the time needed to analyze complex research data. Reliance on press releases and news conferences gives voice primarily to commercial viewpoints, yet news organizations are under pressure to get news out to the public quickly on limited budgets. And there is also the lurking possibility of pressure from medical advertisers to present the news in ways that are consistent with their goals.
A middle ground is needed between simply passing on sensationalized, commercially generated versions of “breakthroughs” in medical science and undertaking time-consuming investigative reporting. Without this middle ground, the news media cannot fulfill their responsibility to the public to present well-reported and fair analyses of medical issues. When this is not possible, journalists have an obligation to let readers, listeners and viewers know the constraints that have prevented them from presenting such an assessment of the news.
Hopefully, these eight principles will assist journalists in their efforts to reach such ground and offer a fair and accurate picture of breaking medical news. Whether this is an achievable goal—within the customary practices of journalism today—is another story.
John Abramson, M.D., completed a two-year Robert Wood Johnson fellowship before practicing family medicine for 20 years in Hamilton, Massachusetts. He served as the chair of the Department of Family Practice at Lahey Clinic and is a clinical instructor at Harvard Medical School. He is currently working full time on a book about the causes and consequences of the crisis in American health care.
