0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Research Letters |

The Frequency of Medical Reversal FREE

Vinay Prasad, MD; Victor Gall, MD; Adam Cifu, MD
[+] Author Affiliations

Author Affiliations: Departments of Medicine, Northwestern University Feinberg School of Medicine (Drs Prasad and Gall), and University of Chicago (Dr Cifu), Chicago, Illinois.


Arch Intern Med. 2011;171(18):1675-1676. doi:10.1001/archinternmed.2011.295.
Text Size: A A A
Published online

We use the term reversal to signify the phenomenon of a new trial—superior to predecessors because of better design, increased power, or more appropriate controls—contradicting current clinical practice. In recent years, a number of such reversals have occurred. Use of hormone therapy,1 the class 1C antiarrhythmic agents,2 and the pulmonary artery catheter3 have decreased when trials demonstrated that they are either less effective than previously thought or harmful. Reversal not only affects medications and diagnostic tests. Previously accepted indications for surgical and medical procedures have also been contradicted. In 2007, the Clinical Outcomes Utilizing Revascularization and Aggressive Drug Evaluation (COURAGE)4 trial found no benefit to support percutaneous coronary intervention (vs optimal medical therapy) in many patients with stable coronary artery disease, an indication that was previously accepted. The implications of reversal are notable. Reversal implies error or harm to patients who underwent the practice in question, during the years it was considered effective. Reversal also undermines trust in the medical system. We sought to estimate the frequency of reversal by examining 1 year of original publications in the New England Journal of Medicine.

Other researchers have studied the rate of reversal in medical research.5 Studies of medical interventions are often followed by studies that either reach the opposite result or suggest the magnitude of effect was initially overestimated. Among high-citation count publications, Ioannidis5 found that 16% were contradicted by future studies, and another 16% were found to have smaller effects than initially thought. Herein, we focused on existing practices that were contradicted in a given period in high-impact literature. Knowing the rate of, and predisposing factors for, reversal may have implications for the approval of medical therapies.

We reviewed all Original Articles in the New England Journal of Medicine in 2009 (the last complete year of the publication at the time of our investigation). Articles were classified on the basis of whether they addressed a medical practice, whether that practice that was new or already in place, and whether the studies' results were positive or negative. Two reviewers independently classified these articles (V.P. and V.G.). This yielded a highly similar profile (weighted Cohen κ = 0.94). Where there was disagreement, a third reviewer (A.C.) adjudicated those discrepancies. Next, we studied the precondition(s) that permitted reversal in each case. Two reviewers independently articulated the precondition (V.P. and A.C.), and these results were combined. This again yielded a highly similar profile (weighted Cohen κ = 0.85).

There were 212 original articles published in the New England Journal of Medicine in 2009, 124 (58%) of which made some claim with respect to a medical practice. The remainder was predominantly descriptive, molecular science publications. Of these 124 articles, 89 (72%) investigated a new medical practice, while 35 (28%) studied a practice already in adoption; 91 (73%) were randomized controlled trials; 19 (15%) were prospective cohort studies; 13 (10%) were retrospective cohort; and 1 was a case-control study. Of the 124 studies, 82 (66%) reported positive results and 42 (33%) reported negative findings; 61 (49%) reported a new practice surpassing current care; 12 (10%) reported a new practice failing to improve on current practice; 16 (13%) reported an existing practice that was upheld as beneficial and 16 (13%) constituted reversal; and 19 (15%) were classified as inconclusive.

The eFigure details all 16 reversals that appeared in 2009, and how each article contradicted current medical practice. Reversals included medical therapies (prednisone use among preschool-aged children with viral wheezing, tight glycemic control in intensive care unit patients, and the routine use of statins in hemodialysis patients), invasive procedures (endoscopic vein harvesting for coronary artery bypass graft surgery and percutaneous coronary intervention for chronic total artery occlusions and atherosclerotic renal artery disease), and screening tests. In several cases, current guidelines were contradicted by the study in question, as indicated in the third column of the eFigure.

The Figure is an attempt to identify the underlying reason that permitted reversal. Confidence in physiologic models as the prime reason to adopt a practice initially was the most common precondition for reversal.

Place holder to copy figure label and caption
Graphic Jump Location

Figure. A table of reversals and why we erred initially.

The reversal of medical practice is not uncommon in high-impact literature: 13% of articles that make a claim about a medical practice constituted reversal in our review of 1 year of the New England Journal of Medicine. The range of reversals we encountered is broad and encompasses many arenas of medical practice including screening tests and all types of therapeutics.

One may argue that not all the cases we examined are truly reversals. Newer studies, though generally more robust than their predecessors, may not necessarily be correct. However, on average, better-controlled and better-powered studies do provide stronger truth claims.6 Given the quality of studies published in the New England Journal of Medicine, we believe that the results reported are more likely to be enduring. The reversal of medical practice is an important subject with far-reaching consequences. Further study is necessary and of profound importance.

Correspondence: Dr Cifu, University of Chicago Medical Center, 5841 S Maryland Ave, Chicago, IL 60637 (adamcifu@uchicago.edu).

Published Online: July 11, 2011. doi:10.1001/archinternmed.2011.295

Author Contributions: Dr Prasad had full access to the data and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Prasad and Cifu. Acquisition of data: Prasad and Gall. Analysis and interpretation of data: Prasad, Gall, and Cifu. Drafting of the manuscript: Prasad. Critical revision of the manuscript for important intellectual content: Prasad, Gall, and Cifu. Statistical analysis: Prasad. Administrative, technical, and material support: Gall. Study supervision: Cifu.

Financial Disclosure: None reported.

Rossouw JE, Anderson GL, Prentice RL,  et al; Writing Group for the Women's Health Initiative Investigators.  Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results from the Women's Health Initiative randomized controlled trial.  JAMA. 2002;288(3):321-333
PubMed   |  Link to Article
Echt DS, Liebson PR, Mitchell LB,  et al.  Mortality and morbidity in patients receiving encainide, flecainide, or placebo: the Cardiac Arrhythmia Suppression Trial.  N Engl J Med. 1991;324(12):781-788
PubMed   |  Link to Article
Binanay C, Califf RM, Hasselblad V,  et al; ESCAPE Investigators and ESCAPE Study Coordinators.  Evaluation study of congestive heart failure and pulmonary artery catheterization effectiveness: the ESCAPE Trial.  JAMA. 2005;294(13):1625-1633
PubMed   |  Link to Article
Boden WE, O’Rourke RA, Teo KK,  et al; COURAGE Trial Research Group.  Optimal medical therapy with or without PCI for stable coronary disease.  N Engl J Med. 2007;356(15):1503-1516
PubMed   |  Link to Article
Ioannidis JP. Contradicted and initially stronger effects in highly cited clinical research.  JAMA. 2005;294(2):218-228
PubMed   |  Link to Article
Ioannidis JP. Why most published research findings are false [published online August 30, 2005].  PLoS Med. 2005;2(8):e124
PubMed   |  Link to Article

Figures

Place holder to copy figure label and caption
Graphic Jump Location

Figure. A table of reversals and why we erred initially.

Tables

References

Rossouw JE, Anderson GL, Prentice RL,  et al; Writing Group for the Women's Health Initiative Investigators.  Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results from the Women's Health Initiative randomized controlled trial.  JAMA. 2002;288(3):321-333
PubMed   |  Link to Article
Echt DS, Liebson PR, Mitchell LB,  et al.  Mortality and morbidity in patients receiving encainide, flecainide, or placebo: the Cardiac Arrhythmia Suppression Trial.  N Engl J Med. 1991;324(12):781-788
PubMed   |  Link to Article
Binanay C, Califf RM, Hasselblad V,  et al; ESCAPE Investigators and ESCAPE Study Coordinators.  Evaluation study of congestive heart failure and pulmonary artery catheterization effectiveness: the ESCAPE Trial.  JAMA. 2005;294(13):1625-1633
PubMed   |  Link to Article
Boden WE, O’Rourke RA, Teo KK,  et al; COURAGE Trial Research Group.  Optimal medical therapy with or without PCI for stable coronary disease.  N Engl J Med. 2007;356(15):1503-1516
PubMed   |  Link to Article
Ioannidis JP. Contradicted and initially stronger effects in highly cited clinical research.  JAMA. 2005;294(2):218-228
PubMed   |  Link to Article
Ioannidis JP. Why most published research findings are false [published online August 30, 2005].  PLoS Med. 2005;2(8):e124
PubMed   |  Link to Article

Correspondence

CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Please click the checkbox indicating that you have read the full article in order to submit your answers.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.
Submit a Comment

Multimedia

The Frequency of Medical Reversal
Arch Intern Med.2011;171(18):1675-1676.eSupplement

eSupplement -Download PDF (405 KB). This file requires Adobe Reader®.

eFigure. (Figure 1) A table of all reversal in the NEJM in 2009 and how they contradicted existing medical practice

eReferences
Supplemental Content

Some tools below are only available to our subscribers or users with an online account.

Web of Science® Times Cited: 33

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Collections