0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Review Article |

Effects of Computerized Physician Order Entry and Clinical Decision Support Systems on Medication Safety:  A Systematic Review FREE

Rainu Kaushal, MD, MPH; Kaveh G. Shojania, MD; David W. Bates, MD, MSc
[+] Author Affiliations

From the Division of General Internal Medicine, Brigham and Women's Hospital, Partners HealthCare System, and Harvard Medical School, Boston, Mass (Drs Kaushal and Bates); and the Department of Medicine, University of California San Francisco (Dr Shojania). Drs Kaushal and Shojania have no relevant financial interest in this article.


Arch Intern Med. 2003;163(12):1409-1416. doi:10.1001/archinte.163.12.1409.
Text Size: A A A
Published online

Background  Iatrogenic injuries related to medications are common, costly, and clinically significant. Computerized physician order entry (CPOE) and clinical decision support systems (CDSSs) may reduce medication error rates.

Methods  We identified trials that evaluated the effects of CPOE and CDSSs on medication safety by electronically searching MEDLINE and the Cochrane Library and by manually searching the bibliographies of retrieved articles. Studies were included for systematic review if the design was a randomized controlled trial, a nonrandomized controlled trial, or an observational study with controls and if the measured outcomes were clinical (eg, adverse drug events) or surrogate (eg, medication errors) markers. Two reviewers extracted all the data. Discussion resolved any disagreements.

Results  Five trials assessing CPOE and 7 assessing isolated CDSSs met the criteria. Of the CPOE studies, 2 demonstrated a marked decrease in the serious medication error rate, 1 an improvement in corollary orders, 1 an improvement in 5 prescribing behaviors, and 1 an improvement in nephrotoxic drug dose and frequency. Of the 7 studies evaluating isolated CDSSs, 3 demonstrated statistically significant improvements in antibiotic-associated medication errors or adverse drug events and 1 an improvement in theophylline-associated medication errors. The remaining 3 studies had nonsignificant results.

Conclusions  Use of CPOE and isolated CDSSs can substantially reduce medication error rates, but most studies have not been powered to detect differences in adverse drug events and have evaluated a small number of "homegrown" systems. Research is needed to evaluate commercial systems, to compare the various applications, to identify key components of applications, and to identify factors related to successful implementation of these systems.

MEDICATION ERRORS and adverse drug events (ADEs) are common, costly, and clinically important problems.17 Each year, an estimated 770 000 people are injured or die in hospitals from ADEs, which are injuries resulting from drug use.4,5,8 Adult hospital incidence rates of ADEs have ranged from 2 to 7 per 100 admissions,2,4,9,10 although determination of a precise national estimate is difficult because studies have used varying definitions.11 Approximately 28% of ADEs are associated with a medication error and therefore are judged to be preventable.2 Of preventable ADEs, 56% occurred during drug ordering.2

Two inpatient studies, 1 in adults2 and 1 in children,7 found that medication errors occurred at rates of more than 5% and that approximately half of all medication errors occurred at the stage of drug ordering. The principal types of medication errors include missing a dose and incorrect medication doses, frequencies, or routes.2 The frequency and type of medication errors found depend on the method used to detect them. Other studies,1 which used a direct observation method to assess how accurately orders are carried out, found high rates of drug administration errors.

Analysis of medication errors suggests that prevention strategies targeting systems rather than individuals are most effective in reducing errors.12 Computerized physician order entry (CPOE) and clinical decision support systems (CDSSs) are promising interventions that target the ordering stage of medications, where most medication errors and preventable ADEs occur. Despite growing public mandates and the obvious theoretical advantages of these systems, organizational adoption of CPOE and CDSSs has been limited. The much publicized Institute of Medicine report To Err Is Human13 drew attention to this "digital divide" between health care and other industrial sectors and called for more widespread adoption of information technology solutions to improve medication safety. Results of recent surveys14,15 of US hospitals indicate that only 4.3% to 15.0% of hospitals have an electronic medication order-entry system in place. If partially implemented systems and institutions in the process of acquiring systems are included, this figure may be as high as 30%.14,15 The degree to which health care has lagged behind other sectors in automating complex and hazardous processes is striking. For example, adoption of information technology interventions such as bar coding and automated drug delivery systems to reduce drug administration errors has also been slow. Therefore, we undertook this study to systematically review the cumulative evidence on the effects of CPOE and CDSSs on medication safety.

DEFINITION OF CPOE AND CDSSs

Computerized physician order entry refers to a variety of computer-based systems that share the common features of automating the medication ordering process and that ensure standardized, legible, and complete orders. Clinical decision support systems are built into almost all CPOE systems to varying degrees. Basic clinical decision support provides computerized advice regarding drug doses, routes, and frequencies, and more sophisticated CDSSs can perform drug allergy checks, drug–laboratory value checks, and drug-drug interaction checks and can provide reminders about corollary orders (eg, prompting the user to order glucose checks after ordering insulin) or drug guidelines.16

Clinical decision support systems may also be implemented without CPOE. Basic CDSSs often assist in tasks such as drug selection, dosing, and duration, and more refined CDSSs can incorporate patient- or pathogen-specific information. The ordering physician may view such advice and then proceed with a conventional handwritten medication order.

STUDY IDENTIFICATION AND SELECTION

Studies were identified by searching the US National Library of Medicine MEDLINE electronic bibliographic database and the electronic Cochrane Library. The MEDLINE search strategy was performed using the following MeSH terms: hospital information systems; decision support systems, clinical; and drug therapy, computer-assisted. In addition, we searched for key title words related to computerized order entry and combined the results of these searches with MeSH terms capturing adverse events and medical errors: medical error, iatrogenic disease, sentinel surveillance, and safety. The Cochrane Library was searched using similar key terms and title words. Reference lists from all relevant articles, including 2 systematic reviews,17,18 were reviewed to identify additional primary studies.

Specifically, we sought articles describing computerized systems for performing general order entry or CDSSs for guiding physicians in the order-writing process. Computerized programs that screen for potential ADEs were not included, unless they interact with users during the order-writing process,10,19,20 and neither were CDSSs built into programmable intravenous infusion pumps.2123 Although both of these practices play a role in improving medication safety, they do not affect the stage of order writing, which is the focus of this review.

STUDY EVALUATION

Two of us (R.K. and K.G.S.) reviewed all the articles to determine the level of evidence for practice effectiveness using frameworks developed by the University of California San Francisco–Stanford Evidence-Based Practice Center for the evaluation of study design (Table 1) and measured outcomes (Table 2).24 This classification scheme was developed because of the heterogeneous nature of the studies evaluating CPOE and CDSSs. The scheme incorporates features of existing frameworks and recommendations for evaluating and synthesizing evidence.2531 We included articles with a minimum level 3 study design (observational studies with controls) and level 2 outcomes (surrogate clinical outcomes). Studies that reported a mixture of level 2 and level 3 outcomes (outcomes with an indirect or unestablished connection to the target safety outcome) were included, as had been decided prospectively. Disagreements were resolved by discussion.

Table Graphic Jump LocationTable 1. Hierarchy of Study Designs*
Table Graphic Jump LocationTable 2. Hierarchy of Outcome Measures
OUTCOME DEFINITIONS

Medication errors are errors in the process of ordering, transcribing, dispensing, administering, or monitoring medications. One example is an order written for acetaminophen without a route of administration. Medication errors include a mixture of errors with differing potentials for patient injury.

Potential ADEs are medication errors with significant potential to harm a patient that may or may not actually reach a patient. An example of an intercepted potential ADE is an order written for a morphine overdose that is noticed and corrected by a pharmacist before the drug is administered. An example of a nonintercepted potential ADE is an administered overdose of morphine to a patient who does not have any sequelae. Medication errors and potential ADEs were considered surrogate outcomes (level 2).

Adverse drug events are injuries resulting from drug use and therefore constitute clinical outcomes (level 1). Adverse drug events associated with a medication error are considered preventable, whereas those not associated with a medication error are considered nonpreventable. An example of a preventable ADE is the development of a rash after the administration of penicillin to a known penicillin-allergic patient. In contrast, a nonpreventable ADE is the development of a penicillin-associated rash in a patient with no known previous allergies.

Nonintercepted serious medication errors include nonintercepted potential ADEs and preventable ADEs (ie, medication errors that either have the potential to or actually cause harm to a patient). Errors that are not intercepted and have the potential to or actually cause injury are the most important from the perspective of patient safety.

DATA EXTRACTION AND ANALYSIS

Studies were grouped into 2 categories: those evaluating CPOE with CDSSs and those evaluating CDSSs alone. No studies were found assessing CPOE alone. We did not quantitatively score the quality of the studies owing to the recognized difficulties in quality scoring in general29,32 and especially for a heterogeneous group of studies such as those included in this review. Nonetheless, both reviewers abstracted each study for prospectively determined elements pertaining to methodological quality.33 In addition, the reviewers described the study design, setting, outcomes, and results. Other extracted information included data regarding potential harm from the practice and issues regarding cost and implementation.

STUDY DESIGN AND SETTINGS

Twelve studies met the inclusion criteria for study design and measured outcomes. The 5 studies listed in Table 3 evaluated CPOE with CDSSs.3438 In the first study,34 investigators at the Regenstrief Institute for Health Care (affiliated with the Indiana University School of Medicine, Indianapolis) conducted a randomized controlled trial of 2181 patients evaluating the effects of CPOE on corollary order prescribing. The remaining 4 studies3538 evaluated the CPOE system at Brigham and Women's Hospital (BWH). The first BWH study35 was a cross-sectional analysis of 6771 patients comparing an intervention period of CPOE with CDSSs with a historical period, the next 2 BWH studies36,37 were time series analyses, and the final BWH study38 was a randomized controlled trial with a crossover design of 7490 patients.

Table Graphic Jump LocationTable 3. Studies of Computerized Physician Order Entry (CPOE) With Clinical Decision Support Systems (CDSSs)

Table 4 lists 7 studies3945 that evaluated isolated CDSSs; 6 were randomized controlled trials and 1 was a prospective before-after analysis. All of these studies were conducted in the inpatient setting. Burton et al39 assessed the use of a computerized aminoglycoside dosing program for 75 patients at the Dallas Veterans Affairs Medical Center in Texas. Evans et al,40,41 at LDS Hospital, Salt Lake City, Utah, performed 2 studies on antibiotic CDSSs: a randomized controlled trial40 of empiric antibiotic drug selection using CDSSs with 451 patients and a cross-sectional analysis41 comparing an intervention period of a computer-assisted anti-infective drug management program with a historical control period for 1136 patients in the intensive care unit. Casner et al42 and Hurley et al43 performed randomized controlled trials of computerized theophylline dosing programs with 17 and 48 patients, respectively. Mungall et al44 evaluated a heparin dosing program for 25 inpatients, and White et al45 evaluated a warfarin dosing program for 39 inpatients. In summary, many of the CDSS studies had small sample sizes and consequently were underpowered.

Table Graphic Jump LocationTable 4. Studies of Clinical Decision Support Systems (CDSSs)
STUDY OUTCOMES

Most of the included CPOE studies primarily measured level 2 and level 3 outcomes because level 1 outcomes are significantly less frequent. Therefore, larger and longer studies are necessary to measure the effects of an intervention on ADE rates, and the costs of such studies are very high. The first 2 BWH studies35,36 primarily measured nonintercepted serious medication errors (level 2) and medication errors (level 2) but also included ADEs as a secondary outcome (level 1). The other 3 CPOE studies reported level 2 and level 3 outcomes (ie, prescribing practices,37 corollary orders,34 and appropriate drug dose and frequency38). Corollary orders (level 2 and level 3 outcomes) are orders needed to detect or ameliorate potential effects of a trigger order, for example, ordering regular laboratory tests of coagulation status after starting a patient on intravenous heparin therapy.

Similarly, the studies evaluating CDSSs report level 1 and level 2 outcomes, with level 1 outcomes often a secondary end point. Burton et al39 reported rates of toxic serum aminoglycoside levels (level 2). Evans et al determined rates of pathogen susceptibility to an antibiotic drug regimen (level 2)40 and rates of anti-infective drug–associated ADEs (level 1).41 Casner et al42 and Hurley et al43 reported rates of toxic serum theophylline levels (level 2). Mungall et al44 and White et al45 reported bleeding complications (level 1), and White et al45 also reported overanticoagulation rates (level 2).

CPOE AND MEDICATION SAFETY

The first BWH study35 assessing the impact of CPOE with CDSSs demonstrated a 55% decrease in nonintercepted serious medication errors (P = .01). As a secondary outcome, this study found a 17% decrease in the preventable ADE rate, which was not statistically significant (P = .37). The CPOE application at the time of this study included only basic decision support, with limited checking for allergies and drug-drug interactions. The second study,36 a time series analysis, evaluated medication error rates before CPOE and in the 3 years subsequent to its implementation. It demonstrated an 81% decrease in medication errors and an 86% decrease in nonintercepted serious medication errors (P<.001 for both). This study found a decrease in the rate of ADEs per 1000 patient-days from 14.7 to 9.6 during the study (P = .09) and a decrease in the number of preventable ADEs from 5 to 2 (P = .05).

The remaining 3 studies assessed more specific types of medication errors. Overhage et al34 demonstrated a greater than 25% improvement in the rates of corollary orders with implementation of computerized reminders. Teich et al37 demonstrated 5 prescribing improvements in types, doses, and frequencies of drug use with the implementation of computerized clinical decision support. Finally, Chertow et al38 demonstrated a 13% decrease in inappropriate dose and a 24% decrease in inappropriate frequency for nephrotoxic drugs in patients with renal insufficiency (P<.001 for both).

CDSSs AND MEDICATION SAFETY

Three of the studies assessing isolated CDSSs evaluated computerized antibiotic drug advice and demonstrated lower rates of toxic levels, improved pathogen susceptibility, and a decreased anti-infective drug–associated ADE rate. Burton et al39 evaluated a computerized aminoglycoside dosing program and demonstrated lower rates of toxic levels in intervention patients, but the results were not statistically significant (P = .40). Evans et al40 demonstrated a 17% greater pathogen susceptibility to an antibiotic drug regimen suggested by a computer consultant vs a physician (P<.001). In another study, Evans et al41 reported a 70% decrease in ADEs caused by anti-infective agents through use of a computer-based anti-infective drug management program (P = .02).

Two other studies evaluated theophylline dosing. Casner et al42 demonstrated no difference in rates of toxic serum levels. In contrast, Hurley et al43 demonstrated significantly lower rates of toxic levels in intervention patients (18.9%) than in control patients (37.8%) (P = .04).

The final 2 studies evaluated anticoagulation agents. Evaluation of a heparin dosing system demonstrated lower rates of bleeding events in intervention patients (4.2%) vs control patients (7.7%), but without statistical significance (P = .6).44 Similarly, evaluation of a warfarin (Coumadin) dosing program demonstrated lower rates of bleeding complications (0% vs 8%) and overanticoagulation rates (5% vs 17%), but neither result was statistically significant (P = .11).45

These studies provide evidence that the use of CPOE with CDSSs significantly decreases medication error and serious medication error rates at 2 institutions with "homegrown" systems. However, the effect on ADE rates has not been adequately tested because studies with sufficient power have not been performed. There is a strong correlation between medication errors and ADEs, so such applications will almost certainly reduce ADE rates. Nevertheless, medication errors have widely varying potential for harm, and it seems easiest to prevent those that rarely cause injury.35,36

Some of the CDSS studies,40,41,43 particularly those evaluating antibiotic drug–associated programs, demonstrated focal reductions in medication errors with statistical significance as well as some decreases in ADE rates. Although other CDSSs tended to have statistically insignificant results, these studies39,42,44,45 were underpowered, with sample sizes of 17 to 75 patients. Comprehensive applications, including CPOE and sophisticated decision support, will likely have the greatest effect.

One important question is whether the currently available data are sufficiently compelling that CPOE should be widely adopted or whether further research is required. We believe that further studies targeted at a few critical questions are desirable but not a requirement before widespread adoption. For example, a multicenter study evaluating the impact of CPOE on ADE rates would cost tens of millions of dollars and would be hard to perform because CPOE is a complex application touching on so many parts of the clinical and information systems. Instead, research should focus on questions such as the following: What are the differences among various CPOE systems? What are barriers to adoption? What are the key decision support elements? How effective are specific pieces of decision support? How should these applications be implemented in community hospitals?

Most studies of CPOE have assessed only 2 internally developed (homegrown) systems. To date, dissemination of these systems has been limited for a variety of reasons. Most hospitals use commercial systems. Relatively few vendors have CPOE applications that have broad use at more than a handful of hospitals. As with evaluations of therapeutic agents, there is a reasonable expectation of a class effect46 with many CPOE systems, but classes of CPOE remain to be established. For example, CPOE systems with no decision support will almost certainly decrease error rates less than systems with sophisticated decision support. Thus, one area for further research consists of developing tools to assess the extent to which a specific commercial CPOE application will reduce the medication error rate or the preventable ADE rate. Comparisons among such commercial products will likewise be informative.

Organizational adoption of CPOE has been limited. One survey14 of 668 hospitals indicated that 15% had at least partially implemented CPOE. A more recent survey15 of pharmacy directors at 1091 acute care hospitals in the United States (49% response rate) reported that 4.3% of hospitals had an electronic medication order-entry system in place.

Many barriers to CPOE adoption exist. Rogers47 suggests that perceived attributes of an innovation and organizational social context strongly affect the innovation adoption rate. Perhaps most important, a health care institution must garner financial and organizational support before introducing CPOE with CDSSs. Computerized physician order entry requires large up-front capital investment with more remote, albeit substantial, returns. Such investment is especially challenging when organizations are losing money. In addition to the financial obstacles, implementing sophisticated new clinical information systems presents substantial organizational challenges owing to the impact on institutional culture and clinical workflow4851 and the need to accommodate existing institutional systems used for billing, laboratory, and pharmacy data.52

In addition, the efficacy of individual decision support elements warrants further investigation. Many of the CDSS studies included in this review produced nonsignificant results. Yet, it is difficult to draw definitive conclusions because of the small sample sizes. Larger studies need to be performed, as do studies identifying key, successful decision support elements.

COSTS AND BENEFITS OF CPOE AND CDSSs

Purchasing commercial CPOE systems is generally more expensive than is internally developing systems. Brigham and Women's Hospital has reported costs of $1.9 million for developing and implementing CPOE in 1992, with ongoing maintenance costs of $500 000 per year, although this was incremental to what was already a highly developed clinical system.53 Fewer data are available regarding the cost of purchasing and implementing large commercial systems, but it may be on the order of tens of millions of dollars, especially if related clinical applications such as a clinical data repository must be upgraded. Several studies34,37,54 report that only minimal resources are needed to introduce or maintain decision support programs into existing order-entry programs.

The beneficial effects of CPOE systems extend beyond medication safety and include reduced costs and quality improvement. These benefits have been achieved by providing feedback about the appropriateness and costs of laboratory and radiologic tests, easy implementation of clinical pathways, improved quality measurement, and improved coding and billing. Brigham and Women's Hospital estimated net savings of $5 to $10 million per year for the CPOE system.55 In a randomized controlled clinical trial, Tierney et al56 demonstrated that CPOE linked to a comprehensive electronic medical record system resulted in charges that were $887 (12.7%) lower per admission. Cost savings associated with averted ADEs may be considerable. For example, BWH, a 720-bed academic institution, estimated costs before CPOE implementation of $2.8 million annually for preventable ADEs.57 Evans et al58 reported a $100 000 per year cost avoidance with a computer-assisted antibiotic drug dosing program attributable to decreased antibiotic drug use and avoided ADEs.

POTENTIAL FOR HARM

As with any other technology, CPOE and CDSSs may introduce different types of medication errors. Incorrect default dosing or route suggestions may lead to potentially erroneous orders. For example, the first time series analysis36 at BWH demonstrated an initial increase in intercepted potential ADEs attributable to the ordering screen structure for potassium chloride, which made it easy to order large doses of intravenous potassium. Once identified, this error was rectified, but this event underscores the importance of ongoing close scrutiny of CPOE and CDSSs. In general, as users become accustomed to CPOE and CDSSs, they are likely to accept computer suggestions with minimal reflection,59 emphasizing the importance of testing decision support default settings and suggestions.

When CPOE systems are not electronically linked to computerized pharmacy systems, pharmacists must manually reenter orders into the pharmacy system, with a resultant increase in chance of error. Pedersen et al15 found that 25.7% of surveyed hospitals with electronic prescribing lacked information system linkages to pharmacy systems.

The trigger level for computerized warnings must be set to the appropriate sensitivity. In situations with a potential for significant harm, it is important that providers receive warnings without being overwhelmed by alarms of marginal value. Hardware outages and software instability pose further risks. In particular, the reliability needed for CPOE is much higher than that required for systems that simply report laboratory test results. Finally, physicians can electronically write an order in the wrong patient's record, analogous to handwriting an order in the wrong patient's medical chart.

PRESENT LEGISLATION AND PUBLIC MANDATES

In the meantime, public and private groups are increasingly demanding implementation of CPOE and other information technologies. The Leapfrog Group, a consortium of companies that belong to the Business Roundtable, has endorsed CPOE in hospitals as 1 of 3 changes that would most improve patient safety in the United States.60 A Medicare Payment Advisory Commission report61 suggested instituting financial incentives for CPOE implementation. Legislation has also been introduced at the federal and state levels. United States Senators Bob Graham (D-Fla) and Olympia Snowe (R-Maine) recently introduced a bill, titled the "Medication Errors Reduction Act of 2001," to establish an informatics system grant program for hospitals and skilled nursing facilities.62 At the state level, California recently enacted legislation stipulating that acute-care hospitals implement information technology such as CPOE to reduce medication-related errors.63

In conclusion, use of CPOE and isolated CDSSs significantly decreases medication error rates and provides other important benefits related to medication use. Funding constraints and public pressure to bridge the digital divide in health care make a large trial comparing paper ordering to sophisticated CPOE systems unlikely. However, a need exists for research evaluating commercial systems, the relative benefits of different classes of systems, and factors related to successful implementation of these systems.

Corresponding author and reprints: Rainu Kaushal, MD, MPH, Division of General Internal Medicine, Brigham and Women's Hospital, PBBA3, 75 Francis St, Boston, MA 02115 (e-mail: rkaushal@partners.org).

Accepted for publication August 16, 2002.

Portions of this article are based on work performed by the authors in collaboration with the University of California San Francisco–Stanford Evidence-Based Practice Center under contract 290-97-0013 to the Agency for Healthcare Research and Quality, Rockville, Md.

Dr Bates has received honoraria for speaking from the Eclipsys Corp (Boca Raton, Fla), which has licensed the rights to the BWH Clinical Information System. The hospital no longer has a financial relationship with Eclipsys. Dr Bates is a coinventor on patent No. 6029138 held by BWH on the use of decision support software for medical management, licensed to Medicalis Corp (Boston, Mass). He holds a minority equity position in the privately held company Medicalis, which develops Web-based decision support for radiology test ordering, and he serves as a consultant to Medicalis. He is also a consultant and serves on the advisory board for McKesson MedManagement (Minneapolis, Minn), a company that assists hospitals in preventing adverse drug events.

We thank Robert M. Wachter, MD, Bradford W. Duncan, MD, Kathryn M. McDonald, MSc, and Amy J. Markowitz, JD, for their careful review of early drafts of this manuscript.

The authors are responsible for the contents of this article, including any clinical or treatment recommendations. No statement in this article should be construed as an official position of the Agency for Healthcare Research and Quality or of the US Department of Health and Human Services.

Barker  KNMikeal  RLPearson  REIllig  NAMorse  ML Medication errors in nursing homes and small hospitals. Am J Hosp Pharm. 1982;39987- 991
Bates  DWCullen  DLaird  N  et al.  Incidence of adverse drug events and potential adverse drug events: implications for prevention. JAMA. 1995;27429- 34
Link to Article
Dean  BSAllan  ELBarber  NDBarker  KN Comparison of medication errors in an American and a British hospital. Am J Health Syst Pharm. 1995;522543- 2549
Classen  DCPestotnik  SLEvans  RSLloyd  JFBurke  JP Adverse drug events in hospitalized patients: excess length of stay, extra costs, and attributable mortality. JAMA. 1997;277301- 306
Link to Article
Cullen  DJSweitzer  BJBates  DWBurdick  EEdmondson  ALeape  LL Preventable adverse drug events in hospitalized patients: a comparative study of intensive care units and general care units. Crit Care Med. 1997;251289- 1297
Link to Article
Lesar  TSLomaestro  BMPohl  H Medication-prescribing errors in a teaching hospital: a 9-year experience. Arch Intern Med. 1997;1571569- 1576
Link to Article
Kaushal  RBates  DWLandrigan  C  et al.  Medication errors and adverse drug events in pediatric inpatients. JAMA. 2001;2852114- 2120
Link to Article
Cullen  DJBates  DWSmall  SDCooper  JBNemeskal  ARLeape  LL The incident reporting system does not detect adverse drug events: a problem for quality improvement. Jt Comm J Qual Improv. 1995;21541- 548
Bates  DWBoyle  DLVander Vliet  MBSchneider  JLeape  LL Relationship between medication errors and adverse drug events. J Gen Intern Med. 1995;10199- 205
Link to Article
Jha  AKKuperman  GJTeich  JM  et al.  Identifying adverse drug events: development of a computer-based monitor and comparison with chart review and stimulated voluntary report. J Am Med Inform Assoc. 1998;5305- 314
Link to Article
US General Accounting Office, Adverse Drug Events.  Washington, DC US General Accounting Office2000;Publication No. GAO/HEHS-00-21.
Leape  LLBates  DWCullen  DJ  et al.  Systems analysis of adverse drug events. JAMA. 1995;27435- 43
Link to Article
Institute of Medicine, To Err Is Human: Building a Safer Health System.  Washington, DC National Academy Press1999;
Ash  JSGorman  PNHersh  WR Physician order entry in US hospitals. Proc AMIA Symp. 1998;235- 239
Pedersen  CASchneider  PJSantell  JP ASHP national survey of pharmacy practice in hospital settings: prescribing and transcribing—2001. Am J Health Syst Pharm. 2001;582251- 2266
Bates  DWCohen  MLeape  LLOverhage  JMShabot  MMSheridan  T Reducing the frequency of errors in medicine using information technology. J Am Med Inform Assoc. 2001;8299- 308
Link to Article
Walton  RTHarvey  EDovey  SFreemantle  N Computerised advice on drug dosage to improve prescribing practice. Cochrane Database Syst Rev. 2001;1CD002894
Hunt  DLHaynes  RBHanna  SESmith  K Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA. 1998;2801339- 1346
Link to Article
Raschke  RAGollihare  BWunderlich  TA  et al.  A computer alert system to prevent injury from adverse drug events: development and evaluation in a community teaching hospital. JAMA. 1998;2801317- 1320
Link to Article
Rind  DMSafran  CPhillips  RS  et al.  Effect of computer-based alerts on the treatment and outcomes of hospitalized patients. Arch Intern Med. 1994;1541511- 1517
Link to Article
Theil  DRStanley III  TEWhite  WD  et al.  Midazolam and fentanyl continuous infusion anesthesia for cardiac surgery: a comparison of computer-assisted versus manual infusion systems. J Cardiothorac Vasc Anesth. 1993;7300- 306
Link to Article
Ruiz  RBorches  DGonzalez  ACorral  J A new sodium-nitroprusside-infusion controller for the regulation of arterial blood pressure. Biomed Instrum Technol. 1993;27244- 251
Alvis  JMReves  JGGovier  AV  et al.  Computer-assisted continuous infusions of fentanyl during cardiac anesthesia: comparison with a manual method. Anesthesiology. 1985;6341- 49
Link to Article
Owens  DKNease  RF  Jr A normative analytic framework for development of practice guidelines for specific clinical populations. Med Decis Making. 1997;17409- 426
Link to Article
Harris  RPHelfand  MWoolf  SH  et al.  Current methods of the US Preventive Services Task Force: a review of the process. Am J Prev Med. 2001;20 (suppl) 21- 35
Link to Article
Guyatt  GSchunemann  HCook  D  et al.  Grades of recommendation for antithrombotic agents. Chest. 2001;1193S- 7S
Link to Article
Bucher  HCGuyatt  GHCook  DJHolbrook  AMcAlister  FAfor the Evidence-Based Medicine Working Group, Users' guides to the medical literature, XIX: applying clinical trial results, A: how to use an article measuring the effect of an intervention on surrogate end points. JAMA. 1999;282771- 778
Link to Article
Mulrow  CLanghorne  PGrimshaw  J Integrating heterogeneous pieces of evidence in systematic reviews. Ann Intern Med. 1997;127989- 995
Link to Article
Juni  PWitschi  ABloch  REgger  M The hazards of scoring the quality of clinical trials for meta-analysis. JAMA. 1999;2821054- 1060
Link to Article
McKee  MBritton  ABlack  NMcPherson  KSanderson  CBain  C Methods in health services research: interpreting the evidence: choosing between randomized and non-randomized studies. BMJ. 1999;319312- 315
Link to Article
Stroup  DFBerlin  JAMorton  SC  et al.  Meta-analysis of observational studies in epidemiology: a proposal for reporting. JAMA. 2000;2832008- 2012
Link to Article
Juni  PAltman  DGEgger  M Systematic reviews in health care: assessing the quality of controlled clinical trials. BMJ. 2001;32342- 46
Link to Article
Shojania  KGDuncan  BWMcDonald  KMWachter  RM Making Health Care Safer: A Critical Analysis of Patient Safety Practices.  Rockville, Md Agency for Healthcare Research and Quality2001;AHRQ publication No. 01-E058.
Overhage  JMTierney  WMZhou  XHMcDonald  CJ A randomized trial of "corollary orders" to prevent errors of omission. J Am Med Inform Assoc. 1997;4364- 375
Link to Article
Bates  DWLeape  LLCullen  DJ  et al.  Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA. 1998;2801311- 1316
Link to Article
Bates  DWTeich  JLee  J  et al.  The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc. 1999;6313- 321
Link to Article
Teich  JMMerchia  PRSchmiz  JLKuperman  GJSpurr  CBates  DW Effects of computerized physician order entry in prescribing practices. Arch Intern Med. 2000;1602741- 2747
Link to Article
Chertow  GMLee  JKuperman  GJ  et al.  Guided medication dosing for inpatients with renal insufficiency. JAMA. 2001;2862839- 2844
Link to Article
Burton  MEAsh  CLHill  DP  JrHandy  TShepherd  MDVasko  MR A controlled trial of the cost benefit of computerized bayesian aminoglycoside administration. Clin Pharmacol Ther. 1991;49685- 694
Link to Article
Evans  RSClassen  DCPestonik  SLLundsgaarde  HPBurke  JP Improving empiric antibiotic selection using computer decision support. Arch Intern Med. 1994;154878- 884
Link to Article
Evans  RSPestotnik  SLClassen  DC  et al.  A computer-assisted management program for antibiotics and other antiinfective agents. N Engl J Med. 1998;338232- 238
Link to Article
Casner  PRReilly  RHo  H A randomized controlled trial of computerized pharmacokinetic theophylline dosing versus empiric physician dosing. Clin Pharmacol Ther. 1993;53684- 690
Link to Article
Hurley  SFDziukas  LJMcNeil  JJBrignell  MJ A randomized controlled clinical trial of pharmacokinetic theophylline dosing. Am Rev Respir Dis. 1986;1341219- 1224
Mungall  DRAnbe  DForrester  PL  et al.  A prospective randomized comparison of the accuracy of computer-assisted versus GUSTO nomogram: directed heparin therapy. Clin Pharmacol Ther. 1994;55591- 596
Link to Article
White  RHHong  RVenook  AP  et al.  Initiation of warfarin therapy: comparison of physician dosing with computer-assisted dosing. J Gen Intern Med. 1987;2141- 148
Link to Article
McAlister  FALaupacis  AWells  GASackett  DL Users' guides to the medical literature, XIX: applying clinical trial results, B: guidelines for determining whether a drug is exerting (more than) a class effect. JAMA. 1999;2821371- 1377
Link to Article
Rogers  EM Diffusion of Innovations.  New York, NY Free Press1983;
Lorenzi  NMRiley  RTBlyth  AJSouthon  GDixon  BJ Antecedents of the people and organizational aspects of medical informatics: review of the literature. J Am Med Inform Assoc. 1997;479- 93
Link to Article
Oostendorp  AHoekstra  SAarts  J Exploring factors required for successful implementation of healthcare informatics. Stud Health Technol Inform. 1998;51146- 151
Berg  M Patient care information systems and health care work: a sociotechnical approach. Int J Med Inf. 1999;5587- 101
Link to Article
Massaro  TA Introducing physician order entry at a major academic medical center, I: impact on organizational culture and behavior. Acad Med. 1993;6820- 25
Link to Article
Kleinke  JD Release 0.0: clinical information technology in the real world. Health Aff (Millwood). 1998;1723- 38
Link to Article
Glaser  JTeich  JMKuperman  G Impact of information events on medical care. Proceedings of the 1996 HIMMS Annual Conference Chicago, Ill Healthcare Information and Management Systems Society1996;1- 9
Shojania  KGYokoe  DPlatt  RFiskio  JMa'luf  NBates  DW Reducing vancomycin utilization using a computerized guideline: results of a randomized control trial. J Am Med Inform Assoc. 1998;5554- 562
Link to Article
Teich  JGlaser  JBeckley  RFSteen  EB  et al.  Toward cost-effective, quality care: the Brigham Integrated Computing System. Proceedings From the Second Annual Nicholas E. Davies CPR Recognition Symposium Washington, DC Computer-based Patient Record Institute1996;5- 34
Tierney  WMMiller  MEOverhage  JMMcDonald  CJ Physician inpatient order writing on microcomputer workstations: effects on resource utilization. JAMA. 1993;269379- 383
Link to Article
Bates  DWSpell  NCullen  D  et al.  The costs of adverse drug events in hospitalized patients. JAMA. 1997;277307- 311
Link to Article
Evans  RSPestonik  SLClassen  DCBurke  JP Evaluation of a computer-assisted antibiotic-dose monitor. Ann Pharmacother. 1999;331026- 1031
Link to Article
Nadzam  DMMacklis  RM Promoting patient safety: is technology the solution? Jt Comm J Qual Improv. 2001;27430- 436
Milstein  AGalvin  RSDelbanco  SFSalber  PBuck  CR Improving the safety of health care: the Leapfrog initiative. Eff Clin Pract. 2000;3313- 316
Medicare Payment Advisory Commission, Report to the Congress: Selected Medicare Issues.  Washington, DC Medicare Payment Advisory Commission1999;
Not Available, Medication Errors Reduction Act of 2001 S 824, 107th Cong, 1st Sess (2001).
Not Available, California Senate Bill 1875, Chapter 816 §1339.63 (September 28, 2000).

Figures

Tables

Table Graphic Jump LocationTable 1. Hierarchy of Study Designs*
Table Graphic Jump LocationTable 2. Hierarchy of Outcome Measures
Table Graphic Jump LocationTable 3. Studies of Computerized Physician Order Entry (CPOE) With Clinical Decision Support Systems (CDSSs)
Table Graphic Jump LocationTable 4. Studies of Clinical Decision Support Systems (CDSSs)

References

Barker  KNMikeal  RLPearson  REIllig  NAMorse  ML Medication errors in nursing homes and small hospitals. Am J Hosp Pharm. 1982;39987- 991
Bates  DWCullen  DLaird  N  et al.  Incidence of adverse drug events and potential adverse drug events: implications for prevention. JAMA. 1995;27429- 34
Link to Article
Dean  BSAllan  ELBarber  NDBarker  KN Comparison of medication errors in an American and a British hospital. Am J Health Syst Pharm. 1995;522543- 2549
Classen  DCPestotnik  SLEvans  RSLloyd  JFBurke  JP Adverse drug events in hospitalized patients: excess length of stay, extra costs, and attributable mortality. JAMA. 1997;277301- 306
Link to Article
Cullen  DJSweitzer  BJBates  DWBurdick  EEdmondson  ALeape  LL Preventable adverse drug events in hospitalized patients: a comparative study of intensive care units and general care units. Crit Care Med. 1997;251289- 1297
Link to Article
Lesar  TSLomaestro  BMPohl  H Medication-prescribing errors in a teaching hospital: a 9-year experience. Arch Intern Med. 1997;1571569- 1576
Link to Article
Kaushal  RBates  DWLandrigan  C  et al.  Medication errors and adverse drug events in pediatric inpatients. JAMA. 2001;2852114- 2120
Link to Article
Cullen  DJBates  DWSmall  SDCooper  JBNemeskal  ARLeape  LL The incident reporting system does not detect adverse drug events: a problem for quality improvement. Jt Comm J Qual Improv. 1995;21541- 548
Bates  DWBoyle  DLVander Vliet  MBSchneider  JLeape  LL Relationship between medication errors and adverse drug events. J Gen Intern Med. 1995;10199- 205
Link to Article
Jha  AKKuperman  GJTeich  JM  et al.  Identifying adverse drug events: development of a computer-based monitor and comparison with chart review and stimulated voluntary report. J Am Med Inform Assoc. 1998;5305- 314
Link to Article
US General Accounting Office, Adverse Drug Events.  Washington, DC US General Accounting Office2000;Publication No. GAO/HEHS-00-21.
Leape  LLBates  DWCullen  DJ  et al.  Systems analysis of adverse drug events. JAMA. 1995;27435- 43
Link to Article
Institute of Medicine, To Err Is Human: Building a Safer Health System.  Washington, DC National Academy Press1999;
Ash  JSGorman  PNHersh  WR Physician order entry in US hospitals. Proc AMIA Symp. 1998;235- 239
Pedersen  CASchneider  PJSantell  JP ASHP national survey of pharmacy practice in hospital settings: prescribing and transcribing—2001. Am J Health Syst Pharm. 2001;582251- 2266
Bates  DWCohen  MLeape  LLOverhage  JMShabot  MMSheridan  T Reducing the frequency of errors in medicine using information technology. J Am Med Inform Assoc. 2001;8299- 308
Link to Article
Walton  RTHarvey  EDovey  SFreemantle  N Computerised advice on drug dosage to improve prescribing practice. Cochrane Database Syst Rev. 2001;1CD002894
Hunt  DLHaynes  RBHanna  SESmith  K Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA. 1998;2801339- 1346
Link to Article
Raschke  RAGollihare  BWunderlich  TA  et al.  A computer alert system to prevent injury from adverse drug events: development and evaluation in a community teaching hospital. JAMA. 1998;2801317- 1320
Link to Article
Rind  DMSafran  CPhillips  RS  et al.  Effect of computer-based alerts on the treatment and outcomes of hospitalized patients. Arch Intern Med. 1994;1541511- 1517
Link to Article
Theil  DRStanley III  TEWhite  WD  et al.  Midazolam and fentanyl continuous infusion anesthesia for cardiac surgery: a comparison of computer-assisted versus manual infusion systems. J Cardiothorac Vasc Anesth. 1993;7300- 306
Link to Article
Ruiz  RBorches  DGonzalez  ACorral  J A new sodium-nitroprusside-infusion controller for the regulation of arterial blood pressure. Biomed Instrum Technol. 1993;27244- 251
Alvis  JMReves  JGGovier  AV  et al.  Computer-assisted continuous infusions of fentanyl during cardiac anesthesia: comparison with a manual method. Anesthesiology. 1985;6341- 49
Link to Article
Owens  DKNease  RF  Jr A normative analytic framework for development of practice guidelines for specific clinical populations. Med Decis Making. 1997;17409- 426
Link to Article
Harris  RPHelfand  MWoolf  SH  et al.  Current methods of the US Preventive Services Task Force: a review of the process. Am J Prev Med. 2001;20 (suppl) 21- 35
Link to Article
Guyatt  GSchunemann  HCook  D  et al.  Grades of recommendation for antithrombotic agents. Chest. 2001;1193S- 7S
Link to Article
Bucher  HCGuyatt  GHCook  DJHolbrook  AMcAlister  FAfor the Evidence-Based Medicine Working Group, Users' guides to the medical literature, XIX: applying clinical trial results, A: how to use an article measuring the effect of an intervention on surrogate end points. JAMA. 1999;282771- 778
Link to Article
Mulrow  CLanghorne  PGrimshaw  J Integrating heterogeneous pieces of evidence in systematic reviews. Ann Intern Med. 1997;127989- 995
Link to Article
Juni  PWitschi  ABloch  REgger  M The hazards of scoring the quality of clinical trials for meta-analysis. JAMA. 1999;2821054- 1060
Link to Article
McKee  MBritton  ABlack  NMcPherson  KSanderson  CBain  C Methods in health services research: interpreting the evidence: choosing between randomized and non-randomized studies. BMJ. 1999;319312- 315
Link to Article
Stroup  DFBerlin  JAMorton  SC  et al.  Meta-analysis of observational studies in epidemiology: a proposal for reporting. JAMA. 2000;2832008- 2012
Link to Article
Juni  PAltman  DGEgger  M Systematic reviews in health care: assessing the quality of controlled clinical trials. BMJ. 2001;32342- 46
Link to Article
Shojania  KGDuncan  BWMcDonald  KMWachter  RM Making Health Care Safer: A Critical Analysis of Patient Safety Practices.  Rockville, Md Agency for Healthcare Research and Quality2001;AHRQ publication No. 01-E058.
Overhage  JMTierney  WMZhou  XHMcDonald  CJ A randomized trial of "corollary orders" to prevent errors of omission. J Am Med Inform Assoc. 1997;4364- 375
Link to Article
Bates  DWLeape  LLCullen  DJ  et al.  Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA. 1998;2801311- 1316
Link to Article
Bates  DWTeich  JLee  J  et al.  The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc. 1999;6313- 321
Link to Article
Teich  JMMerchia  PRSchmiz  JLKuperman  GJSpurr  CBates  DW Effects of computerized physician order entry in prescribing practices. Arch Intern Med. 2000;1602741- 2747
Link to Article
Chertow  GMLee  JKuperman  GJ  et al.  Guided medication dosing for inpatients with renal insufficiency. JAMA. 2001;2862839- 2844
Link to Article
Burton  MEAsh  CLHill  DP  JrHandy  TShepherd  MDVasko  MR A controlled trial of the cost benefit of computerized bayesian aminoglycoside administration. Clin Pharmacol Ther. 1991;49685- 694
Link to Article
Evans  RSClassen  DCPestonik  SLLundsgaarde  HPBurke  JP Improving empiric antibiotic selection using computer decision support. Arch Intern Med. 1994;154878- 884
Link to Article
Evans  RSPestotnik  SLClassen  DC  et al.  A computer-assisted management program for antibiotics and other antiinfective agents. N Engl J Med. 1998;338232- 238
Link to Article
Casner  PRReilly  RHo  H A randomized controlled trial of computerized pharmacokinetic theophylline dosing versus empiric physician dosing. Clin Pharmacol Ther. 1993;53684- 690
Link to Article
Hurley  SFDziukas  LJMcNeil  JJBrignell  MJ A randomized controlled clinical trial of pharmacokinetic theophylline dosing. Am Rev Respir Dis. 1986;1341219- 1224
Mungall  DRAnbe  DForrester  PL  et al.  A prospective randomized comparison of the accuracy of computer-assisted versus GUSTO nomogram: directed heparin therapy. Clin Pharmacol Ther. 1994;55591- 596
Link to Article
White  RHHong  RVenook  AP  et al.  Initiation of warfarin therapy: comparison of physician dosing with computer-assisted dosing. J Gen Intern Med. 1987;2141- 148
Link to Article
McAlister  FALaupacis  AWells  GASackett  DL Users' guides to the medical literature, XIX: applying clinical trial results, B: guidelines for determining whether a drug is exerting (more than) a class effect. JAMA. 1999;2821371- 1377
Link to Article
Rogers  EM Diffusion of Innovations.  New York, NY Free Press1983;
Lorenzi  NMRiley  RTBlyth  AJSouthon  GDixon  BJ Antecedents of the people and organizational aspects of medical informatics: review of the literature. J Am Med Inform Assoc. 1997;479- 93
Link to Article
Oostendorp  AHoekstra  SAarts  J Exploring factors required for successful implementation of healthcare informatics. Stud Health Technol Inform. 1998;51146- 151
Berg  M Patient care information systems and health care work: a sociotechnical approach. Int J Med Inf. 1999;5587- 101
Link to Article
Massaro  TA Introducing physician order entry at a major academic medical center, I: impact on organizational culture and behavior. Acad Med. 1993;6820- 25
Link to Article
Kleinke  JD Release 0.0: clinical information technology in the real world. Health Aff (Millwood). 1998;1723- 38
Link to Article
Glaser  JTeich  JMKuperman  G Impact of information events on medical care. Proceedings of the 1996 HIMMS Annual Conference Chicago, Ill Healthcare Information and Management Systems Society1996;1- 9
Shojania  KGYokoe  DPlatt  RFiskio  JMa'luf  NBates  DW Reducing vancomycin utilization using a computerized guideline: results of a randomized control trial. J Am Med Inform Assoc. 1998;5554- 562
Link to Article
Teich  JGlaser  JBeckley  RFSteen  EB  et al.  Toward cost-effective, quality care: the Brigham Integrated Computing System. Proceedings From the Second Annual Nicholas E. Davies CPR Recognition Symposium Washington, DC Computer-based Patient Record Institute1996;5- 34
Tierney  WMMiller  MEOverhage  JMMcDonald  CJ Physician inpatient order writing on microcomputer workstations: effects on resource utilization. JAMA. 1993;269379- 383
Link to Article
Bates  DWSpell  NCullen  D  et al.  The costs of adverse drug events in hospitalized patients. JAMA. 1997;277307- 311
Link to Article
Evans  RSPestonik  SLClassen  DCBurke  JP Evaluation of a computer-assisted antibiotic-dose monitor. Ann Pharmacother. 1999;331026- 1031
Link to Article
Nadzam  DMMacklis  RM Promoting patient safety: is technology the solution? Jt Comm J Qual Improv. 2001;27430- 436
Milstein  AGalvin  RSDelbanco  SFSalber  PBuck  CR Improving the safety of health care: the Leapfrog initiative. Eff Clin Pract. 2000;3313- 316
Medicare Payment Advisory Commission, Report to the Congress: Selected Medicare Issues.  Washington, DC Medicare Payment Advisory Commission1999;
Not Available, Medication Errors Reduction Act of 2001 S 824, 107th Cong, 1st Sess (2001).
Not Available, California Senate Bill 1875, Chapter 816 §1339.63 (September 28, 2000).

Correspondence

CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.
Submit a Comment

Multimedia

Some tools below are only available to our subscribers or users with an online account.

Web of Science® Times Cited: 502

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Collections
PubMed Articles
JAMAevidence.com

Users' Guides to the Medical Literature
Chapter 9.6. Clinical Decision Support Systems

The Rational Clinical Examination
Evidence Summary and Review 3