0
Original Investigation | Health Care Reform

Unintended Effects of a Computerized Physician Order Entry Nearly Hard-Stop Alert to Prevent a Drug Interaction:  A Randomized Controlled Trial FREE

Brian L. Strom, MD, MPH; Rita Schinnar, MPA; Faten Aberra, MD, MSCE; Warren Bilker, PhD; Sean Hennessy, PharmD, PhD; Charles E. Leonard, PharmD; Eric Pifer, MD
[+] Author Affiliations

Author Affiliations: Center for Clinical Epidemiology and Biostatistics, Department of Biostatistics and Epidemiology, and Center for Education and Research on Therapeutics (Drs Strom, Bilker, Hennessy, Leonard, and Pifer and Ms Schinnar) and Division of Gastroenterology (Dr Aberra), University of Pennsylvania School of Medicine, Philadelphia. Dr Pifer is now with El Camino Hospital, Mountain View, California.


Arch Intern Med. 2010;170(17):1578-1583. doi:10.1001/archinternmed.2010.324.
Text Size: A A A
Published online

Background  The effectiveness of computerized physician order entry (CPOE) systems has been modest, largely because clinicians frequently override electronic alerts.

Methods  To evaluate the effectiveness of a nearly “hard stop” CPOE prescribing alert intended to reduce concomitant orders for warfarin and trimethoprim-sulfamethoxazole, a randomized clinical trial was conducted at 2 academic medical centers in Philadelphia, Pennsylvania. A total of 1981 clinicians were assigned to either an intervention group receiving a nearly hard stop alert or a control group receiving the standard practice. The study duration was August 9, 2006, through February 13, 2007.

Results  The proportion of desired responses (ie, not reordering the alert-triggering drug within 10 minutes of firing) was 57.2% (111 of 194 hard stop alerts) in the intervention group and 13.5% (20 of 148) in the control group (adjusted odds ratio, 0.12; 95% confidence interval, 0.045-0.33). However, the study was terminated early because of 4 unintended consequences identified among patients in the intervention group: a delay of treatment with trimethoprim-sulfamethoxazole in 2 patients and a delay of treatment with warfarin in another 2 patients.

Conclusions  An electronic hard stop alert as part of an inpatient CPOE system seemed to be extremely effective in changing prescribing. However, this intervention precipitated clinically important treatment delays in 4 patients who needed immediate drug therapy. These results illustrate the importance of formal evaluation and monitoring for unintended consequences of programmatic interventions intended to improve prescribing habits.

Trial Registration  clinicaltrials.gov Identifier: NCT00870298

Figures in this Article

Anticoagulants, especially warfarin, are the cornerstone of therapy for several diseases, including the prophylaxis and treatment of pulmonary embolism, venous thrombosis, and atrial fibrillation with embolization. Although anticoagulants confer significant benefits, they are associated with a high risk of adverse events, specifically bleeding, which is more likely in the setting of supratherapeutic anticoagulation.

Many medications can increase the anticoagulation effects of warfarin. In observational studies1,2 assessing risk factors for overanticoagulation, antibiotics have been a common culprit, with trimethoprim-sulfamethoxazole among the most common. In a retrospective cohort study3 of patients using warfarin who initiated any of several antibiotic therapies, 69% of patients using trimethoprim-sulfamethoxazole exhibited clinically significant elevations in the international normalized ratio to greater than 4. Adverse bleeding events developed in 13% of the patients exposed to trimethoprim-sulfamethoxazole and in none of the other antibiotic groups studied.3 A case-control study4 showed that recent initiation of trimethoprim-sulfamethoxazole therapy in patients receiving warfarin was associated with hospitalization for gastrointestinal bleeding (adjusted odds ratio, 1.46 [95% confidence interval (CI), 1.16-1.85] for a prescription filled 0-5 days before the hospitalization and 2.54 [2.08-3.10] for a prescription filled 6-10 days before the hospitalization). For several years, the Hospital of the University of Pennsylvania has, therefore, had in place a program of pharmacy interventions in which pharmacists telephoned prescribers to notify them of this interaction whenever a simultaneous order was written, that is, a pharmacist intervention program. Yet, some physicians still continued to prescribe these drugs concurrently.

In 1997, the University of Pennsylvania Health System implemented a computerized physician order entry (CPOE) system for inpatient care.5 Yet, numerous previous studies have shown that using preprogrammed alerts contained in the decision support as part of a CPOE system is often ineffective in changing prescribing611 and is frequently related to poor design characteristics of the alerts and insufficient consideration of human factors in the implementation.12,13 The results also may differ when using vendor-developed solutions vs homegrown solutions. Automatic order alerts are built into the CPOEs to protect patients against harmful administration of medications. Automatic order alerts may be either soft or hard. When soft order alerts appear on the computer screen, clinicians are alerted about potential problems associated with the particular prescription order and are presented with alternative treatments available for consideration. In contrast, when hard stop-order alerts appear on the screen, the clinician's order is blocked from further execution to avert potentially serious reactions. The goal of this study was to measure the effectiveness of a customized nearly hard stop alert in reducing concomitant orders for warfarin and trimethoprim-sulfamethoxazole compared with the standard practice of a pharmacist intervention program.

This study was a randomized controlled trial initially planned for 7 months, that was started on August 9, 2006, but was terminated early on February 13, 2007.

SETTING AND PARTICIPANTS

This study was conducted at the Hospital of the University of Pennsylvania and at Penn Presbyterian Medical Center, where all inpatient orders are entered using the Sunrise Clinical Manager (Eclipsys Corp, Atlanta, Georgia) CPOE system. The study participants were 1981 eligible clinicians involved in inpatient care, of whom 1971 were included in the final analysis (1872 resident physicians [RPs] and 99 nurse practitioners [NPs]).

The randomization procedure involved listing in a spreadsheet the names of all the RPs and NPs and assigning each a random number. We then rank-ordered the list based on the random numbers and selected the first half to be in the intervention group and the second half to be in the control group.

INTERVENTION

The intervention included clinicians subject to an automatic electronic hard-stop alert of a trimethoprim-sulfamethoxazole or warfarin order entered into the CPOE system whenever an RP or NP placed an order for trimethoprim-sulfamethoxazole with an already-active warfarin order, if warfarin was ordered for a patient already taking trimethoprim-sulfamethoxazole, or when ordering both simultaneously. The hard-stop alert appeared as a pop-up window that notified the clinician that the order could not be processed because of a significant potential drug-drug interaction. The specific text of the stop alert read as follows:

The prescription of warfarin and TMP/Sulfa together is completely prohibited except in cases of urgent need for the TMP/Sulfa.

If you are attempting to prescribe warfarin and the patient is already on TMP/Sulfa, discontinue the TMP/Sulfa and your order for warfarin will be processed.

If you are attempting to prescribe TMP/Sulfa and feel that your patient has an urgent need, then contact the inpatient pharmacy and you will be directed as to how to process the order for TMP/Sulfa.

Alerts could be overridden in 2 ways. One way, which did not involve pharmacist intervention, was to enter in the order the indication of Pneumocystis carinii pneumonia (PCP) prophylaxis. This triggered an acknowledge alert that read as follows: “Dr. xxx, you have certified that this patient has PCP. Click on ‘Acknowledge’ button to certify that this diagnosis is still active to proceed with this prescription.” The other way to override the alert was to bypass the CPOE altogether by calling the pharmacist directly.

The control group continued with the prevailing practice of the pharmacist telephoning prescribers to notify them of this interaction and recommend cessation for concurrent warfarin and trimethoprim-sulfamethoxazole orders, that is, a pharmacist intervention program.

OUTCOMES AND FOLLOW-UP

The primary outcome was a new concurrent prescription order for trimethoprim-sulfamethoxazole and warfarin accepted through the electronic ordering system. In this context, not reordering the alert-triggering drug within 10 minutes after the alert fired (or would have fired in the control group) represented a desired response by the providers. This time frame was chosen because it was a reasonable period within which a clinician, having just received an alert, would have reacted to it. However, because 10 minutes was an arbitrary time frame, we also repeated the analysis using an outcome of 24 hours; the results were comparable and are not presented.

Two potential adverse outcomes of the computerized hard-stop alert were also of interest. The first was a delay in obtaining trimethoprim-sulfamethoxazole when the practitioner believed that an infection was best treated with trimethoprim-sulfamethoxazole and when the potential warfarin interaction was judged less important than the need for the antibiotic. The second was unintentional warfarin cessation in a patient previously undergoing long-term warfarin therapy. Therefore, the study also assessed the incidence of warfarin cessation on the day when an order of trimethoprim-sulfamethoxazole was attempted in a patient already receiving warfarin.

DATA COLLECTION

All the data were obtained from the CPOE systems of the 2 study hospitals. The unit of analysis for this study was the hard stop alerts that fired, or would have fired, whenever a prescription order for concurrent trimethoprim-sulfamethoxazole and warfarin was encountered during the inpatient stay. Alerts were attributed to the study group of the clinician who ordered the add-on prescription that triggered the alert.

Some alerts appeared to be firing multiple times, immediately one after another, presumably because providers attempted to reenter the order for the triggering medication to attempt to overrule the alert. Accordingly, we applied a 5-minute rule in defining units of analysis; we counted alerts firing within 5 minutes of each other as a single episode and alerts firing more than 5 minutes apart as separate episodes.

HUMAN SUBJECTS REVIEW/DATA MONITORING

This study was reviewed by the University of Pennsylvania's institutional review board (IRB), receiving approval with exemption from obtaining the clinicians' consent and a waiver of Health Insurance Portability and Accountability Act authorization. All faculty and clinical staff at the University of Pennsylvania Health System receive annual notification that they may become part of research evaluating interventions designed to improve clinical practice, specifically, interventions involving electronic systems.

The IRB initially had concerns that depriving the control group of the intervention would be unethical and, in fact, was initially reluctant to approve the study. Another IRB concern was that the new alert could lead to delays in patient care: either that clinicians would discontinue warfarin because of the alert or that the alternative antibiotic for trimethoprim-sulfamethoxazole may not work as well in a given patient. Also, there may be reasons to give trimethoprim-sulfamethoxazole even with warfarin. To ensure that the risk remained minimal, the IRB requested a monitoring plan to look continuously for signals of delaying treatment (ie, inappropriate delay or stoppage of warfarin and inappropriate delay or stoppage of trimethoprim-sulfamethoxazole) resulting from the new alert. Accordingly, a monitoring committee was convened and charged with reviewing monthly reports of all events in which the electronic intervention was activated to determine the occurrences of inappropriate discontinuation or failure to begin therapy in situations in which warfarin and trimethoprim-sulfamethoxazole were appropriate.

STATISTICAL ANALYSIS

The proportions of “desired responses” (ie, not reordering the alert-triggering drug within 10 minutes after alert firing) in the intervention and control groups were compared using logistic regression,14 accounting for clustering15 of RPs and NPs (ie, the nonindependence of outcomes in patients within the same clinic), using STATA version 10 (StataCorp LP, College Station, Texas).

Figure 1 shows the profile of the 1981 clinicians randomized to the 2 study groups (995 intervention clinicians receiving the active hard stop alerts and 986 control clinicians who would have received the alerts and did not), of whom 1971 were included in the final analysis after 10 were excluded because they erroneously appeared in both lists. During the study period, these providers ordered 8826 prescriptions through the CPOE system (3167 warfarin and 1036 trimethoprim-sulfamethoxazole prescriptions ordered by intervention clinicians, and 3630 warfarin and 993 trimethoprim-sulfamethoxazole prescriptions ordered by control clinicians). Seven patients in the intervention group were exempt because of PCP prophylaxis.

Place holder to copy figure label and caption
Figure 1.

Profile of the randomized clinical trial for the warfarin and trimethoprim-sulfamethoxazole study. UPHS indicates University of Pennsylvania Health System. *Duplicates resulted from the same clinicians working in both study hospitals and having different IDs in each hospital but representing the same persons. None received alerts.

Graphic Jump Location

Overall, 437 alerts had fired. Of these, 342 alerts (194 in the intervention group and 148 in the control group) were analyzed as unique events after applying the 5-minute rule. Of the 194 hard stop alerts issued to the intervention group, the proportion of desired responses by the clinicians (ie, not reordering the alert-triggering drug within 10 minutes of firing) was 57.2% (n = 111). Of the 83 undesired responses in the intervention group, the alert-triggering drug ordered was warfarin in 78 and trimethoprim-sulfamethoxazole in 5. In contrast, the comparable proportion in the control group was 13.5% (n = 20). Of the 128 undesired responses in the control group, the alert-triggering drug ordered was warfarin in 121, trimethoprim-sulfamethoxazole in 2, and both drugs simultaneously in 5. Clinicians in the intervention group were less likely than control clinicians to reorder the alert-triggering drug after adjusting for provider type (RP or NP) as a confounder and accounting for clustering by provider (adjusted odds ratio, 0.12; 95% CI, 0.045-0.33). The unadjusted odds ratio was 0.12 (95% CI, 0.07-0.20). Adjustment for hospital was not possible because 1 hospital had only 1 observation in which the alerting medication was reordered within 10 minutes after the stop alert had fired. Note the high mean number of alerts per provider (3.53 in the intervention group and 3.29 in the control group), even after counting alerts triggered by repeated orders within 5 minutes as a single alert event. The greatest proportion of desired responses was observed in the first 3 months of the intervention, after which it steadily declined, suggesting that the effectiveness of the alert may have started to wear off (Figure 2). The groups remained different, however, at the end of the study.

Place holder to copy figure label and caption
Figure 2.

Monthly time trend for the mean proportion of desired responses (ie, not reordering the alert-triggering drug within 10 minutes after alert firing) for each study group. Error bars represent 95% confidence intervals.

Graphic Jump Location

Finally, the study was stopped early by the IRB because of 4 instances of unintended consequences identified in the intervention group. For a synopsis of the monthly monitoring of unintended consequences associated with the intervention—either a delay of treatment with trimethoprim-sulfamethoxazole when determined to be necessary for treatment or inadvertent warfarin discontinuation—see the Table. Review of the electronic records of all 4 individuals involved in these events showed that in no case could we identify specific infectious or thrombotic complications that could have been related to the delays in therapy in the adverse event reporting.

Table Graphic Jump LocationTable. Monthly Monitoring of Unintended Consequences of the Stop Alert for Warfarin and Trimethoprim-Sulfamethoxazole

This study revealed that an electronic order intervention that is nearly a hard-stop alert can, indeed, be effective in reducing the undesired prescribing, at least initially. This intervention also resulted in delay in ordering trimethoprim-sulfamethoxazole when the practitioners believed that trimethoprim-sulfamethoxazole was necessary for their patients. This necessitated the early termination of this study for ethical reasons because of potential for harm in the intervention arm, a dramatic finding given the IRB's initial concern for the potential for harm by having a control group that was denied the intervention.

Alerts are typically communicated through pop-up warning messages on the computer screen (ie, simple information triggers). Such CPOE system interventions have reduced medication error rates,16,17 but even important electronic alerts are sometimes overridden,611 suggesting that there is still much to learn about the best way to present such alerts to providers.10,18,19 In inpatient settings, Hsieh et al20 found that only 20% of physicians changed prescriptions in response to drug allergy alerts, and Payne et al11 found that only 12% of critical drug interaction alerts and 31% of drug allergy interaction alerts resulted in a changed prescription order. The principal reasons for low response by clinicians to the automated alerts are many low-consequence alert firings and poor design of alerts.12,13 Clinicians tend to override the alerts because they are perceived to be nonspecific and lack the clinicians' additional knowledge of the clinical situation for the specific patient context.12,2125 Given the perception among clinicians that most alerts are inappropriate and are a nuisance because of low clinical relevance, CPOE-based drug alerts may perhaps be more effective if they were customized to include only a limited number of critically important alerts.12,24,25 Another approach has been to design alerts that require from the clinician a specific response to the alert (eg, entering a reason for overriding).20,2629 Alerts that require a response seem to be somewhat more effective,12,2629 presumably because they engage the clinician's attention, although not always,20,29 and they increase the clinician's time burden.18,20

Still another approach is to design hard stop alerts that prevent the clinician from completing the prescription order entry, although some find it objectionable on the grounds that decision support should not replace the clinician's responsibility for their patients.10 Published reports of interventions that used hard stop alerts are scant.13

An important component of CPOE systems is to check prescription orders by clinicians to reduce medication errors and improve patient safety. As part of this support, CPOE systems are designed to trigger safety alerts to prevent adverse drug reactions. However, excessive alerting can disrupt clinician workflows,18,30 and overdependence on CPOE systems may cause chaos when the system is down, with clinicians having trouble remembering medication contraindications, standard dosages, and hospital formulary recommendations.10,30,31 Such unintended consequences5,32 and unintended therapy delays, such as occurred in a few cases in this study that were determined to be definitely or probably related to the alert intervention, provide yet another example of the need for formal evaluations of CPOE interventions rather than assuming that they are always necessarily of benefit.

Because many prescribing “errors” detected by automated systems may, in fact, represent reasonable prescribing decisions, it is proper to inquire about the clinical appropriateness of the simultaneous warfarin and trimethoprim-sulfamethoxazole orders that were entered despite the intervention. The review of these cases deemed them to represent delays in appropriate therapy being administered. One was a failure to prescribe appropriate trimethoprim-sulfamethoxazole prophylaxis for an otherwise critically ill patient. Another was a delay in initiation of antibiotic therapy recommended by the Infectious Diseases Consultation Service. The others were delays in initiation of warfarin therapy.

As noted in a recent review by Eslami et al,17 many trials of CPOE and medication safety were not randomized. The strength of this study was its randomized design and statistical power. It is generally accepted that randomization of at least 100 subjects will produce balance between the study groups,33 and, of course, the present sample size is much larger than this.

The decision to randomize clinicians rather than patients was motivated by several considerations. First, because the order is written by clinicians, it is appropriate to consider each order by them as an opportunity for error. Second, each medical practitioner has a unique access code to use the electronic ordering system, and the order system menu can be varied by individual user. In addition, we wanted to keep each practitioner in the same study group for the duration of the study to minimize contamination between the 2 groups. However, there is the possibility that the difference in effect that we observed between the intervention and control groups for concurrent orders of trimethoprim-sulfamethoxazole and warfarin may have diminished across time during the study because RPs and NPs usually work in teams and may work alongside other physicians who may be assigned to a different group in the study. It is common for RPs and NPs to discuss a patient's care plan, which includes issues of ordering medications. Across time, the RPs and NPs from both groups may have learned not to order trimethoprim-sulfamethoxazole with an active order of warfarin because of awareness of the hard-stop alert. We attempted to reduce contamination by trying to complete this study as rapidly as possible. It was initially planned to last 7 months but had to be terminated early.

Finally, because we did not review the medical records of all the patients who, in fact, went on to receive the concurrent warfarin and trimethoprim-sulfamethoxazole medications and because we did not follow up on possible adverse events associated with this drug interaction in these patients, we do not have information to conclude that the benefits of reducing the critical warfarin and trimethoprim-sulfamethoxazole drug interaction outweigh the harms that were observed in this trial. Yet, the harm here was believed to be sufficient that the IRB terminated the study early.

In conclusion, we showed that a computerized decision support intervention—a nearly hard-stop alert—was markedly effective in reducing the prescribing of an undesired drug-drug combination. However, this intervention had unintended adverse consequences that were deemed sufficiently serious to warrant discontinuation of the intervention and early termination of the study. This emphasizes the need for formal evaluation and monitoring of programmatic interventions rather than simply assuming that they will be effective.

Correspondence: Brian L. Strom, MD, MPH, Center for Clinical Epidemiology and Biostatistics, University of Pennsylvania School of Medicine, 824 Blockley Hall, 423 Guardian Dr, Philadelphia, PA 19104-6021 (bstrom@exchange.upenn.edu).

Accepted for Publication: March 20, 2010.

Author Contributions:Study concept and design: Strom, Schinnar, Aberra, Hennessy, and Pifer. Acquisition of data: Strom and Pifer. Analysis and interpretation of data: Strom, Schinnar, Aberra, Bilker, Hennessy, Leonard, and Pifer. Drafting of the manuscript: Strom and Schinnar. Critical revision of the manuscript for important intellectual content: Aberra, Bilker, Hennessy, Leonard, and Pifer. Statistical analysis: Bilker. Obtained funding: Strom. Administrative, technical, and material support: Schinnar and Leonard. Study supervision: Strom.

Financial Disclosure: None reported.

Funding/Support: This study was funded in part by the University of Pennsylvania Health System and in part by cooperative agreement U18HS016946 from the Agency for Healthcare Research and Quality.

Disclaimer: The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.

Previous Presentation: This study was presented at the 25th International Conference on Pharmacoepidemiology & Therapeutic Risk Management; August 18, 2009; Providence, Rhode Island.

Additional Contributions: Todd Barton, MD; Jeffrey Staab, MD, MS; and Robert Gross, MD, MSCE, University of Pennsylvania IRB Oversight Committee provided valuable suggestions and guidance; Jesse Chittams, MS; Asaf Hanish, MPH; Qing Liu, BS; and Catherine T. Williams-Smith, BS, Biostatistics Analysis Center of the Center for Clinical Epidemiology and Biostatistics, University of Pennsylvania School of Medicine, created the analytic data set; Maggie Massary, MS, University of Pennsylvania Health System, helped develop the electronic alert and guided us through the elements of the electronic intervention and Sunrise data collection; and Anne Saint John, AAS, assisted with manuscript preparation.

Panneerselvam  SBaglin  CLefort  WBaglin  T Analysis of risk factors for over-anticoagulation in patients receiving long-term warfarin. Br J Haematol 1998;103 (2) 422- 424
PubMed
Howard  PAEllerbeck  EFEngelman  KKPatterson  KL The nature and frequency of potential warfarin drug interactions that increase the risk of bleeding in patients with atrial fibrillation. Pharmacoepidemiol Drug Saf 2002;11 (7) 569- 576
PubMed
Glasheen  JJFugit  RVProchazka  AV The risk of overanticoagulation with antibiotic use in outpatients on stable warfarin regimens. J Gen Intern Med 2005;20 (7) 653- 656
PubMed
Schelleman  HBilker  WBBrensinger  CMHan  XKimmel  SEHennessy  S Warfarin with fluoroquinolones, sulfonamides, or azole antifungals: interactions and the risk of hospitalization for gastrointestinal bleeding. Clin Pharmacol Ther 2008;84 (5) 581- 588
PubMed
Koppel  RMetlay  JPCohen  A  et al.  Role of computerized physician order entry systems in facilitating medication errors. JAMA 2005;293 (10) 1197- 1203
PubMed
Feldstein  ACSmith  DHPerrin  N  et al.  Reducing warfarin medication interactions: an interrupted time series evaluation. Arch Intern Med 2006;166 (9) 1009- 1015
PubMed
Lapane  KLWaring  MESchneider  KLDubé  CQuilliam  BJ A mixed method study of the merits of e-prescribing drug alerts in primary care. J Gen Intern Med 2008;23 (4) 442- 446
PubMed
Judge  JField  TSDeFlorio  M  et al.  Prescribers' responses to alerts during medication ordering in the long term care setting. J Am Med Inform Assoc 2006;13 (4) 385- 390
PubMed
Weingart  SNToth  MSands  DZAronson  MDDavis  RBPhillips  RS Physicians' decisions to override computerized drug alerts in primary care. Arch Intern Med 2003;163 (21) 2625- 2631
PubMed
van der Sijs  HAarts  JVulto  ABerg  M Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc 2006;13 (2) 138- 147
PubMed
Payne  THNichol  WPHoey  PSavarino  J Characteristics and override rates of order checks in a practitioner order entry system. Proc AMIA Symp 2002;602- 606
PubMed
Shah  NRSeger  ACSeger  DL  et al.  Improving acceptance of computerized prescribing alerts in ambulatory care. J Am Med Inform Assoc 2006;13 (1) 5- 11
PubMed
Paterno  MDMaviglia  SMGorman  PN  et al.  Tiering drug-drug interaction alerts by severity increases compliance rates. J Am Med Inform Assoc 2009;16 (1) 40- 46
PubMed
McCullagh  PNelder  JA Generalized Linear Models. 2nd ed. London, England Chapman & Hall Ltd1983;
Thompson  SGPSPyke  SDHardy  RJ The design and analysis of paired cluster randomized trials: an application of meta-analysis techniques. Stat Med 1997;16 (18) 2063- 2079
PubMed
Shamliyan  TADuval  SDu  JKane  RL Just what the doctor ordered. Review of the evidence of the impact of computerized physician order entry system on medication errors. Health Serv Res 2008;43 (1, pt 1) 32- 53
PubMed
Eslami  Sde Keizer  NFAbu-Hanna  A The impact of computerized physician medication order entry in hospitalized patients—a systematic review. Int J Med Inform 2008;77 (6) 365- 376
PubMed
Miller  RAWaitman  LRChen  SRosenbloom  ST The anatomy of decision support during inpatient care provider order entry (CPOE): empirical observations from a decade of CPOE experience at Vanderbilt. J Biomed Inform 2005;38 (6) 469- 485
PubMed
van der Sijs  HLammers  Lvan den Tweel  A  et al.  Time-dependent drug-drug interaction alerts in care provider order entry: software may inhibit medication error reductions. J Am Med Inform Assoc 2009;16 (6) 864- 868
PubMed
Hsieh  TCKuperman  GJJaggi  T  et al.  Characteristics and consequences of drug allergy alert overrides in a computerized physician order entry system. J Am Med Inform Assoc 2004;11 (6) 482- 491
PubMed
Tamblyn  RHuang  ATaylor  L  et al.  A randomized trial of the effectiveness of on-demand versus computer-triggered drug decision support in primary care. J Am Med Inform Assoc 2008;15 (4) 430- 438
PubMed
Taylor  LKTamblyn  R Reasons for physician non-adherence to electronic drug alerts. Stud Health Technol Inform 2004;107 (pt 2) 1101- 1105
PubMed
Glassman  PABelperio  PSimon  BLanto  ALee  M Exposure to automated drug alerts over time: effects on clinicians' knowledge and perceptions. Med Care 2006;44 (3) 250- 256
PubMed
Kuperman  GJBobb  APayne  TH  et al.  Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc 2007;14 (1) 29- 40
PubMed
Humphries  TLCarroll  NChester  EAMagid  DRocho  B Evaluation of an electronic critical drug interaction program coupled with active pharmacist intervention. Ann Pharmacother 2007;41 (12) 1979- 1985
PubMed
Kucher  NKoo  SQuiroz  R  et al.  Electronic alerts to prevent venous thromboembolism among hospitalized patients. N Engl J Med 2005;352 (10) 969- 977
PubMed
Litzelman  DKDittus  RSMiller  METierney  WM Requiring physicians to respond to computerized reminders improves their compliance with preventive care protocols. J Gen Intern Med 1993;8 (6) 311- 317
PubMed
Peterson  JFKuperman  GJShek  CPatel  MAvorn  JBates  DW Guided prescription of psychotropic medications for geriatric inpatients. Arch Intern Med 2005;165 (7) 802- 807
PubMed
Grizzle  AJMahmood  MHKo  Y  et al.  Reasons provided by prescribers when overriding drug-drug interaction alerts. Am J Manag Care 2007;13 (10) 573- 578
PubMed
Ash  JSSittig  DFPoon  EGGuappone  KCampbell  EDykstra  RH The extent and importance of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc 2007;14 (4) 415- 423
PubMed
Campbell  EMSittig  DFAsh  JSGuappone  KPDykstra  RH Types of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc 2006;13 (5) 547- 556
PubMed
Reckmann  MHWestbrook  JIKoh  YLo  CDay  RO Does computerized provider order entry reduce prescribing errors for hospital inpatients? A systematic review. J Am Med Inform Assoc 2009;16 (5) 613- 623
PubMed
Chow  SCLiu  JP Randomization and Blinding: Design and Analysis of Clinical Trials: Concepts and Methodologies. 2nd ed. Hoboken, NJ John Wiley & Sons Inc2005;135

Figures

Place holder to copy figure label and caption
Figure 1.

Profile of the randomized clinical trial for the warfarin and trimethoprim-sulfamethoxazole study. UPHS indicates University of Pennsylvania Health System. *Duplicates resulted from the same clinicians working in both study hospitals and having different IDs in each hospital but representing the same persons. None received alerts.

Graphic Jump Location
Place holder to copy figure label and caption
Figure 2.

Monthly time trend for the mean proportion of desired responses (ie, not reordering the alert-triggering drug within 10 minutes after alert firing) for each study group. Error bars represent 95% confidence intervals.

Graphic Jump Location

Tables

Table Graphic Jump LocationTable. Monthly Monitoring of Unintended Consequences of the Stop Alert for Warfarin and Trimethoprim-Sulfamethoxazole

References

Panneerselvam  SBaglin  CLefort  WBaglin  T Analysis of risk factors for over-anticoagulation in patients receiving long-term warfarin. Br J Haematol 1998;103 (2) 422- 424
PubMed
Howard  PAEllerbeck  EFEngelman  KKPatterson  KL The nature and frequency of potential warfarin drug interactions that increase the risk of bleeding in patients with atrial fibrillation. Pharmacoepidemiol Drug Saf 2002;11 (7) 569- 576
PubMed
Glasheen  JJFugit  RVProchazka  AV The risk of overanticoagulation with antibiotic use in outpatients on stable warfarin regimens. J Gen Intern Med 2005;20 (7) 653- 656
PubMed
Schelleman  HBilker  WBBrensinger  CMHan  XKimmel  SEHennessy  S Warfarin with fluoroquinolones, sulfonamides, or azole antifungals: interactions and the risk of hospitalization for gastrointestinal bleeding. Clin Pharmacol Ther 2008;84 (5) 581- 588
PubMed
Koppel  RMetlay  JPCohen  A  et al.  Role of computerized physician order entry systems in facilitating medication errors. JAMA 2005;293 (10) 1197- 1203
PubMed
Feldstein  ACSmith  DHPerrin  N  et al.  Reducing warfarin medication interactions: an interrupted time series evaluation. Arch Intern Med 2006;166 (9) 1009- 1015
PubMed
Lapane  KLWaring  MESchneider  KLDubé  CQuilliam  BJ A mixed method study of the merits of e-prescribing drug alerts in primary care. J Gen Intern Med 2008;23 (4) 442- 446
PubMed
Judge  JField  TSDeFlorio  M  et al.  Prescribers' responses to alerts during medication ordering in the long term care setting. J Am Med Inform Assoc 2006;13 (4) 385- 390
PubMed
Weingart  SNToth  MSands  DZAronson  MDDavis  RBPhillips  RS Physicians' decisions to override computerized drug alerts in primary care. Arch Intern Med 2003;163 (21) 2625- 2631
PubMed
van der Sijs  HAarts  JVulto  ABerg  M Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc 2006;13 (2) 138- 147
PubMed
Payne  THNichol  WPHoey  PSavarino  J Characteristics and override rates of order checks in a practitioner order entry system. Proc AMIA Symp 2002;602- 606
PubMed
Shah  NRSeger  ACSeger  DL  et al.  Improving acceptance of computerized prescribing alerts in ambulatory care. J Am Med Inform Assoc 2006;13 (1) 5- 11
PubMed
Paterno  MDMaviglia  SMGorman  PN  et al.  Tiering drug-drug interaction alerts by severity increases compliance rates. J Am Med Inform Assoc 2009;16 (1) 40- 46
PubMed
McCullagh  PNelder  JA Generalized Linear Models. 2nd ed. London, England Chapman & Hall Ltd1983;
Thompson  SGPSPyke  SDHardy  RJ The design and analysis of paired cluster randomized trials: an application of meta-analysis techniques. Stat Med 1997;16 (18) 2063- 2079
PubMed
Shamliyan  TADuval  SDu  JKane  RL Just what the doctor ordered. Review of the evidence of the impact of computerized physician order entry system on medication errors. Health Serv Res 2008;43 (1, pt 1) 32- 53
PubMed
Eslami  Sde Keizer  NFAbu-Hanna  A The impact of computerized physician medication order entry in hospitalized patients—a systematic review. Int J Med Inform 2008;77 (6) 365- 376
PubMed
Miller  RAWaitman  LRChen  SRosenbloom  ST The anatomy of decision support during inpatient care provider order entry (CPOE): empirical observations from a decade of CPOE experience at Vanderbilt. J Biomed Inform 2005;38 (6) 469- 485
PubMed
van der Sijs  HLammers  Lvan den Tweel  A  et al.  Time-dependent drug-drug interaction alerts in care provider order entry: software may inhibit medication error reductions. J Am Med Inform Assoc 2009;16 (6) 864- 868
PubMed
Hsieh  TCKuperman  GJJaggi  T  et al.  Characteristics and consequences of drug allergy alert overrides in a computerized physician order entry system. J Am Med Inform Assoc 2004;11 (6) 482- 491
PubMed
Tamblyn  RHuang  ATaylor  L  et al.  A randomized trial of the effectiveness of on-demand versus computer-triggered drug decision support in primary care. J Am Med Inform Assoc 2008;15 (4) 430- 438
PubMed
Taylor  LKTamblyn  R Reasons for physician non-adherence to electronic drug alerts. Stud Health Technol Inform 2004;107 (pt 2) 1101- 1105
PubMed
Glassman  PABelperio  PSimon  BLanto  ALee  M Exposure to automated drug alerts over time: effects on clinicians' knowledge and perceptions. Med Care 2006;44 (3) 250- 256
PubMed
Kuperman  GJBobb  APayne  TH  et al.  Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc 2007;14 (1) 29- 40
PubMed
Humphries  TLCarroll  NChester  EAMagid  DRocho  B Evaluation of an electronic critical drug interaction program coupled with active pharmacist intervention. Ann Pharmacother 2007;41 (12) 1979- 1985
PubMed
Kucher  NKoo  SQuiroz  R  et al.  Electronic alerts to prevent venous thromboembolism among hospitalized patients. N Engl J Med 2005;352 (10) 969- 977
PubMed
Litzelman  DKDittus  RSMiller  METierney  WM Requiring physicians to respond to computerized reminders improves their compliance with preventive care protocols. J Gen Intern Med 1993;8 (6) 311- 317
PubMed
Peterson  JFKuperman  GJShek  CPatel  MAvorn  JBates  DW Guided prescription of psychotropic medications for geriatric inpatients. Arch Intern Med 2005;165 (7) 802- 807
PubMed
Grizzle  AJMahmood  MHKo  Y  et al.  Reasons provided by prescribers when overriding drug-drug interaction alerts. Am J Manag Care 2007;13 (10) 573- 578
PubMed
Ash  JSSittig  DFPoon  EGGuappone  KCampbell  EDykstra  RH The extent and importance of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc 2007;14 (4) 415- 423
PubMed
Campbell  EMSittig  DFAsh  JSGuappone  KPDykstra  RH Types of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc 2006;13 (5) 547- 556
PubMed
Reckmann  MHWestbrook  JIKoh  YLo  CDay  RO Does computerized provider order entry reduce prescribing errors for hospital inpatients? A systematic review. J Am Med Inform Assoc 2009;16 (5) 613- 623
PubMed
Chow  SCLiu  JP Randomization and Blinding: Design and Analysis of Clinical Trials: Concepts and Methodologies. 2nd ed. Hoboken, NJ John Wiley & Sons Inc2005;135

Correspondence

CME
Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.
NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).
Submit a Comment

Multimedia

Some tools below are only available to our subscribers or users with an online account.

Web of Science® Times Cited: 32

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Topics
PubMed Articles
JAMAevidence.com

Users' Guides to the Medical Literature
Clinical Resolution

Users' Guides to the Medical Literature
Clinical Scenario