Author Affiliations: Departments of Medicine, Northwestern University Feinberg School of Medicine (Drs Prasad and Gall), and University of Chicago (Dr Cifu), Chicago, Illinois.
We use the term reversal to signify the phenomenon of a new trial—superior to predecessors because of better design, increased power, or more appropriate controls—contradicting current clinical practice. In recent years, a number of such reversals have occurred. Use of hormone therapy,1 the class 1C antiarrhythmic agents,2 and the pulmonary artery catheter3 have decreased when trials demonstrated that they are either less effective than previously thought or harmful. Reversal not only affects medications and diagnostic tests. Previously accepted indications for surgical and medical procedures have also been contradicted. In 2007, the Clinical Outcomes Utilizing Revascularization and Aggressive Drug Evaluation (COURAGE)4 trial found no benefit to support percutaneous coronary intervention (vs optimal medical therapy) in many patients with stable coronary artery disease, an indication that was previously accepted. The implications of reversal are notable. Reversal implies error or harm to patients who underwent the practice in question, during the years it was considered effective. Reversal also undermines trust in the medical system. We sought to estimate the frequency of reversal by examining 1 year of original publications in the New England Journal of Medicine.
Other researchers have studied the rate of reversal in medical research.5 Studies of medical interventions are often followed by studies that either reach the opposite result or suggest the magnitude of effect was initially overestimated. Among high-citation count publications, Ioannidis5 found that 16% were contradicted by future studies, and another 16% were found to have smaller effects than initially thought. Herein, we focused on existing practices that were contradicted in a given period in high-impact literature. Knowing the rate of, and predisposing factors for, reversal may have implications for the approval of medical therapies.
We reviewed all Original Articles in the New England Journal of Medicine in 2009 (the last complete year of the publication at the time of our investigation). Articles were classified on the basis of whether they addressed a medical practice, whether that practice that was new or already in place, and whether the studies' results were positive or negative. Two reviewers independently classified these articles (V.P. and V.G.). This yielded a highly similar profile (weighted Cohen κ = 0.94). Where there was disagreement, a third reviewer (A.C.) adjudicated those discrepancies. Next, we studied the precondition(s) that permitted reversal in each case. Two reviewers independently articulated the precondition (V.P. and A.C.), and these results were combined. This again yielded a highly similar profile (weighted Cohen κ = 0.85).
There were 212 original articles published in the New England Journal of Medicine in 2009, 124 (58%) of which made some claim with respect to a medical practice. The remainder was predominantly descriptive, molecular science publications. Of these 124 articles, 89 (72%) investigated a new medical practice, while 35 (28%) studied a practice already in adoption; 91 (73%) were randomized controlled trials; 19 (15%) were prospective cohort studies; 13 (10%) were retrospective cohort; and 1 was a case-control study. Of the 124 studies, 82 (66%) reported positive results and 42 (33%) reported negative findings; 61 (49%) reported a new practice surpassing current care; 12 (10%) reported a new practice failing to improve on current practice; 16 (13%) reported an existing practice that was upheld as beneficial and 16 (13%) constituted reversal; and 19 (15%) were classified as inconclusive.
The eFigure details all 16 reversals that appeared in 2009, and how each article contradicted current medical practice. Reversals included medical therapies (prednisone use among preschool-aged children with viral wheezing, tight glycemic control in intensive care unit patients, and the routine use of statins in hemodialysis patients), invasive procedures (endoscopic vein harvesting for coronary artery bypass graft surgery and percutaneous coronary intervention for chronic total artery occlusions and atherosclerotic renal artery disease), and screening tests. In several cases, current guidelines were contradicted by the study in question, as indicated in the third column of the eFigure.
The Figure is an attempt to identify the underlying reason that permitted reversal. Confidence in physiologic models as the prime reason to adopt a practice initially was the most common precondition for reversal.
Figure. A table of reversals and why we erred initially.
The reversal of medical practice is not uncommon in high-impact literature: 13% of articles that make a claim about a medical practice constituted reversal in our review of 1 year of the New England Journal of Medicine. The range of reversals we encountered is broad and encompasses many arenas of medical practice including screening tests and all types of therapeutics.
One may argue that not all the cases we examined are truly reversals. Newer studies, though generally more robust than their predecessors, may not necessarily be correct. However, on average, better-controlled and better-powered studies do provide stronger truth claims.6 Given the quality of studies published in the New England Journal of Medicine, we believe that the results reported are more likely to be enduring. The reversal of medical practice is an important subject with far-reaching consequences. Further study is necessary and of profound importance.
Correspondence: Dr Cifu, University of Chicago Medical Center, 5841 S Maryland Ave, Chicago, IL 60637 (firstname.lastname@example.org).
Published Online: July 11, 2011. doi:10.1001/archinternmed.2011.295
Author Contributions: Dr Prasad had full access to the data and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Prasad and Cifu. Acquisition of data: Prasad and Gall. Analysis and interpretation of data: Prasad, Gall, and Cifu. Drafting of the manuscript: Prasad. Critical revision of the manuscript for important intellectual content: Prasad, Gall, and Cifu. Statistical analysis: Prasad. Administrative, technical, and material support: Gall. Study supervision: Cifu.
Financial Disclosure: None reported.
Thank you for submitting a comment on this article. It will be reviewed by JAMA Internal Medicine editors. You will be notified when your comment has been published. Comments should not exceed 500 words of text and 10 references.
Do not submit personal medical questions or information that could identify a specific patient, questions about a particular case, or general inquiries to an author. Only content that has not been published, posted, or submitted elsewhere should be submitted. By submitting this Comment, you and any coauthors transfer copyright to the journal if your Comment is posted.
* = Required Field
Disclosure of Any Conflicts of Interest*
Indicate all relevant conflicts of interest of each author below, including all relevant financial interests, activities, and relationships within the past 3 years including, but not limited to, employment, affiliation, grants or funding, consultancies, honoraria or payment, speakers’ bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued. If all authors have none, check "No potential conflicts or relevant financial interests" in the box below. Please also indicate any funding received in support of this work. The information will be posted with your response.
Some tools below are only available to our subscribers or users with an online account.
Download citation file:
Web of Science® Times Cited: 31
Customize your page view by dragging & repositioning the boxes below.
Enter your username and email address. We'll send you a link to reset your password.
Enter your username and email address. We'll send instructions on how to reset your password to the email address we have on record.
Athens and Shibboleth are access management services that provide single sign-on to protected resources. They replace the multiple user names and passwords necessary to access subscription-based content with a single user name and password that can be entered once per session. It operates independently of a user's location or IP address. If your institution uses Athens or Shibboleth authentication, please contact your site administrator to receive your user name and password.