The Frequency of Medical Reversal

We use the term reversal to signify the phenomenon of a new trial—superior to predecessors because of better design, increased power, or more appropriate controls—contradicting current clinical practice. In recent years, a number of such reversals have occurred. Use of hormone therapy,¹ the class 1C antiarrhythmic agents,² and the pulmonary artery catheter³ have decreased when trials demonstrated that they are either less effective than previously thought or harmful. Reversal not only affects medications and diagnostic tests. Previously accepted indications for surgical and medical procedures have also been contradicted. In 2007, the Clinical Outcomes Utilizing Revascularization and Aggressive Drug Evaluation (COURAGE)⁴ trial found no benefit to support percutaneous coronary intervention (vs optimal medical therapy) in many patients with stable coronary artery disease, an indication that was previously accepted. The implications of reversal are notable. Reversal implies error or harm to patients who underwent the practice in question, during the years it was considered effective. Reversal also undermines trust in the medical system. We sought to estimate the frequency of reversal by examining 1 year of original publications in the New England Journal of Medicine.

Other researchers have studied the rate of reversal in medical research.⁵ Studies of medical interventions are often followed by studies that either reach the opposite result or suggest the magnitude of effect was initially overestimated. Among high-citation count publications, Ioannidis⁵ found that 16% were contradicted by future studies, and another 16% were found to have smaller effects than initially thought. Herein, we focused on existing practices that were contradicted in a given period in high-impact literature. Knowing the rate of, and predisposing factors for, reversal may have implications for the approval of medical therapies.

We reviewed all Original Articles in the New England Journal of Medicine in 2009 (the last complete year of the publication at the time of our investigation). Articles were classified on the basis of whether they addressed a medical practice, whether that practice that was new or already in place, and whether the studies' results were positive or negative. Two reviewers independently classified these articles (V.P. and V.G.). This yielded a highly similar profile (weighted Cohen κ = 0.94). Where there was disagreement, a third reviewer (A.C.) adjudicated those discrepancies. Next, we studied the precondition(s) that permitted reversal in each case. Two reviewers independently articulated the precondition (V.P. and A.C.), and these results were combined. This again yielded a highly similar profile (weighted Cohen κ = 0.85).

There were 212 original articles published in the New England Journal of Medicine in 2009, 124 (58%) of which made some claim with respect to a medical practice. The remainder was predominantly descriptive, molecular science publications. Of these 124 articles, 89 (72%) investigated a new medical practice, while 35 (28%) studied a practice already in adoption; 91 (73%) were randomized controlled trials; 19 (15%) were prospective cohort studies; 13 (10%) were retrospective cohort; and 1 was a case-control study. Of the 124 studies, 82 (66%) reported positive results and 42 (33%) reported negative findings; 61 (49%) reported a new practice surpassing current care; 12 (10%) reported a new practice failing to improve on current practice; 16 (13%) reported an existing practice that was upheld as beneficial and 16 (13%) constituted reversal; and 19 (15%) were classified as inconclusive.

The eFigure details all 16 reversals that appeared in 2009, and how each article contradicted current medical practice. Reversals included medical therapies (prednisone use among preschool-aged children with viral wheezing, tight glycemic control in intensive care unit patients, and the routine use of statins in hemodialysis patients), invasive procedures (endoscopic vein harvesting for coronary artery bypass graft surgery and percutaneous coronary intervention for chronic total artery occlusions and atherosclerotic renal artery disease), and screening tests. In several cases, current guidelines were contradicted by the study in question, as indicated in the third column of the eFigure.

The Figure is an attempt to identify the underlying reason that permitted reversal. Confidence in physiologic models as the prime reason to adopt a practice initially was the most common precondition for reversal.

The reversal of medical practice is not uncommon in high-impact literature: 13% of articles that make a claim about a medical practice constituted reversal in our review of 1 year of the New England Journal of Medicine. The range of reversals we encountered is broad and encompasses many arenas of medical practice including screening tests and all types of therapeutics.

One may argue that not all the cases we examined are truly reversals. Newer studies, though generally more robust than their predecessors, may not necessarily be correct. However, on average, better-controlled and better-powered studies do provide stronger truth claims.⁶ Given the quality of studies published in the New England Journal of Medicine, we believe that the results reported are more likely to be enduring. The reversal of medical practice is an important subject with far-reaching consequences. Further study is necessary and of profound importance.

Correspondence: Dr Cifu, University of Chicago Medical Center, 5841 S Maryland Ave, Chicago, IL 60637 (adamcifu@uchicago.edu).

Published Online: July 11, 2011. doi:10.1001/archinternmed.2011.295

Author Contributions: Dr Prasad had full access to the data and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Prasad and Cifu. Acquisition of data: Prasad and Gall. Analysis and interpretation of data: Prasad, Gall, and Cifu. Drafting of the manuscript: Prasad. Critical revision of the manuscript for important intellectual content: Prasad, Gall, and Cifu. Statistical analysis: Prasad. Administrative, technical, and material support: Gall. Study supervision: Cifu.