0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Original Investigation |

Physicians' Decisions to Override Computerized Drug Alerts in Primary Care FREE

Saul N. Weingart, MD, PhD; Maria Toth, MD, PhD; Daniel Z. Sands, MD, MPH; Mark D. Aronson, MD; Roger B. Davis, ScD; Russell S. Phillips, MD
[+] Author Affiliations

From the Division of General Medicine and Primary Care, Beth Israel Deaconess Medical Center, and Department of Medicine, Harvard Medical School, Boston, Mass. The authors have no relevant financial interest in this article.


Arch Intern Med. 2003;163(21):2625-2631. doi:10.1001/archinte.163.21.2625.
Text Size: A A A
Published online

Background  Although computerized physician order entry reduces medication errors among inpatients, little is known about the use of this system in primary care.

Methods  We calculated the override rate among 3481 consecutive alerts generated at 5 adult primary care practices that use a common computerized physician order entry system for prescription writing. For detailed review, we selected a random sample of 67 alerts in which physicians did not prescribe an alerted medication and 122 alerts that resulted in a written prescription. We identified factors associated with the physicians' decisions to override a medication alert, and determined whether an adverse drug event (ADE) occurred.

Results  Physicians overrode 91.2% of drug allergy and 89.4% of high-severity drug interaction alerts. In the multivariable analysis using the medical chart review sample (n = 189), physicians were less likely to prescribe an alerted medication if the prescriber was a house officer (odds ratio [OR], 0.26; 95% confidence interval [CI], 0.08-0.84) and if the patient had many drug allergies (OR, 0.70; 95% CI, 0.53-0.93). They were more likely to override alerts for renewals compared with new prescriptions (OR, 17.74; 95% CI, 5.60-56.18). We found no ADEs in cases where physicians observed the alert and 3 ADEs among patients with alert overrides, a nonsignificant difference (P = .55). Physician reviewers judged that 36.5% of the alerts were inappropriate.

Conclusions  Few physicians changed their prescription in response to a drug allergy or interaction alert, and there were few ADEs, suggesting that the threshold for alerting was set too low. Computerized physician order entry systems should suppress alerts for renewals of medication combinations that patients currently tolerate.

Figures in this Article

IN THE Adverse Drug Event Prevention Study, computerized physician order entry (CPOE) prevented up to 84% of medication errors among patients hospitalized at 2 academic medical centers.1 On the strength of this evidence, CPOE was heralded as a hospital "best practice" in medication safety and a litmus test of safe care.2,3 However, little is known about whether CPOE can prevent adverse drug events (ADEs), defined as injuries due to medications, in ambulatory care.

In published studies to date, investigators4,5 reported that rudimentary computerized prescribing at 2 primary care practices resulted in substantially fewer prescription-writing errors than at similar sites without CPOE. Differences were due to legibility and prompting by the automated systems to provide complete prescription information. However, they found no statistically significant difference between ADE rates at computerized and handwritten sites. The lack of a difference was attributed to the small sample size and lack of advanced features such as drug interaction and drug allergy checking at the computerized sites. Many of the preventable ADEs were due to prescription of drugs to which the patient had a known allergy.

Can advanced CPOE features alter physician behavior and reduce ADEs in primary care? The answer depends in part on whether physicians honor the computer's drug alert warning. In inpatient electronic order-writing systems, some clinicians discount, ignore, or circumvent alerts and reminders.6 Clinicians may forfeit the potential benefit of automated decision support if they find the system annoying, unhelpful, or inefficient.7

To assess the potential benefits of computerized prescribing among ambulatory patients, we examined the behavior of general internists with respect to high-severity drug interaction and drug allergy alerts generated by a CPOE system used at adult primary care practices. We hypothesized that most physicians would honor high-severity drug interaction and drug allergy alerts, and that failure to do so would lead to an increase in ADEs. We sought to examine attributes of patient, prescriber, and medication that affected physicians' decisions to override an alert.

STUDY SITE

This study included physicians at 5 adult primary care practices affiliated with Beth Israel Deaconess Medical Center (BIDMC), a Boston teaching hospital. Two sites are hospital-based clinics together employing 35 full- and part-time academic internists and 158 medical house officers. Three sites are community-based practices employing a total of 19 full-time internists. One community site serves an affluent suburban community; another is a hospital-owned multispecialty group practice; the third is a community health center. Attending physicians and house officers at the 5 study sites care for a diverse population of more than 50 000 patients.

CPOE AND THE ONLINE MEDICAL RECORD

Physician informaticists at BIDMC and the Harvard Center for Clinical Computing, Boston, developed a variety of advanced applications for inpatients and outpatients since the 1970s.8 The Online Medical Record (OMR) was introduced in 1989 in hospital-based primary care practices, but its use spread quickly throughout BIDMC and its community sites.

The OMR provides clinicians with features, including note-writing capability; access to information about laboratory, pathology, and radiology reports and a reminder system for routine screening and preventive health measures. The OMR includes a medication sheet that permits clinicians to submit drug allergy information and to enter and print prescriptions. The system also relies on clinicians to keep medication and allergy information up-to-date. Providers have ready electronic access to the presciption history of their patients. Physicians, nurses, and pharmacists may enter drug allergy and intolerance information into the system; the computer prompts the clinician to record the type of reaction and level of certainty. The prescription-writing feature offers prescribers a list of medications and available strengths, with required fields for drug dose, number of pills or units dispensed, and renewals. Clinicians enter the directions as free text. In 2000, 888 clinicians in 67 practices ordered 296 539 new prescriptions using the OMR.8

In February 2000, the OMR prescription-writing program was enhanced to include drug interaction and drug allergy checks for all prescription orders. Allergy alerts were generated by linking a central database that includes information about each patient's drug allergies and intolerances that was entered by nurses, physicians, and pharmacists in the inpatient, ambulatory, and home care settings. The allergy program triggers an alert if the prescription matches the brand or generic name of the drug or allergen group defined by the National Drug Data File of First Databank, Inc, San Bruno, Calif. Drug interaction alerts were generated by checking patients' electronic medication list using rules in the National Drug Data File of First Databank. First Databank identifies 3 drug interaction severity levels. Level 1 alerts indicate the potential for serious or life-threatening injury and are based on substantial empirical evidence. Level 2 alerts indicate the potential for less serious injury. Level 3 alerts, like level 1 alerts, indicate the potential for serious or life-threatening injury, but the evidence of the interaction is less compelling. A single prescription could generate a drug allergy alert and multiple drug interaction alerts.

When an alert is generated, a note appears on the computer screen that identifies the alerted drug, the drug interaction or allergy, and an indication of the severity of drug interactions (level 1 indicates high; level 2, medium; and level 3, low) (Figure 1 and Figure 2). The system also allows the provider to see a detailed, referenced monograph about the alert. To escape from the alert, the prescriber must select deliberately the "override" menu option; the default is to terminate the order.

Place holder to copy figure label and caption
Figure 1.

Sample drug alert notification using the Online Medical Record. Hcl indicates hydrochloride; Celexa, citalopram hydrobromide; SSRI, selective serotonin reuptake inhibitor; trazodone, trazodone hydrochloride; and 5-HT, serotonin.

Graphic Jump Location
Place holder to copy figure label and caption
Figure 2.

Drug allergy and drug interaction alerts in primary care, by physicians' decisions to override. Hcl indicates hydrochloride; SSRI, selective serotonin reuptake inhibitor; trazodone, trazodone hydrochloride; and 5-HT, serotonin.

Graphic Jump Location
STUDY DESIGN AND SAMPLE SELECTION

The study was a retrospective series. The outcome was the primary care physician's decision whether to write a prescription for a medication flagged for a level 1, 2, or 3 drug interaction or a drug allergy alert.

After obtaining institutional review board approval, we downloaded all 7877 drug interaction and drug allergy alerts generated from October 1 through December 31, 2000. This study period provided a 9-month interval from the implementation of the enhanced CPOE system to allow clinicians (other than some house officers or new hires) to become accustomed to the upgrade. The file included patients' names and hospital identification numbers, name of the medication that generated an alert, date of the alert, type of alert (drug interaction or drug allergy), specific drug-drug interaction(s) associated with a medication, severity of the alert (levels 1-3), and prescriber. The file also included information from the hospital's electronic credentialing system such as physician specialty, sex, and year of medical school graduation.

We reviewed initially all 4751 alerts generated by Department of Medicine internists. We found that physicians frequently tried several times to write a prescription for an alerted medication. They viewed a monograph about the alert, exited from the ordering process using an escape keystroke, then attempted to reorder the medication. This behavior occurred in 1270 of the 4751 alerts (26.7%). To avoid double counting, we selected the last in a sequence of escapes or overrides for a particular physician, drug, patient, and date. This yielded 3481 alerts.

In the initial analysis of all 3481 alerts generated by Department of Medicine internists, we calculated how frequently physicians overrode drug allergy and drug interaction alerts. In the subsequent analysis, we selected a subset of 189 alerts for detailed medical chart review. The medical chart review sample was limited to alerts that triggered drug allergy and level 1 drug interaction alerts, which together represented the greatest potential for harm. We randomly selected 122 alerts (31 drug allergy and 91 drug interaction) from 834 alerts in which the physician chose to override the alert and to complete a written prescription (14.6% of the sample). We randomly selected 67 alerts (15 drug allergy and 52 drug interaction) from 92 alerts where the physician did not prescribe an alerted medication (72.8% of the sample).

MEDICAL CHART REVIEW

An internist (M.T.) reviewed the electronic medical record for the 3-month period from the date an alert was triggered. The reviewer completed a medical chart review instrument and printed the patient's medication sheet and the office note from the date of the index visit. She abstracted demographic information, including the patient's age, sex, and type of insurance. She indicated whether a prescription for the alerted drug was in fact written, the type of encounter (telephone or office visit), and whether the prescribing physician commented in the medical record about the alert. She assessed whether the prescription was for a new medication, a new dose of an existing prescription, or a renewal. She identified the presence of possible ADEs. She documented the name and number of drug allergies, the number of prescription medications, and the number of items on the patient's OMR problem list. A second internist (S.N.W.) reviewed the completed medical chart review instruments and related materials for completeness and accuracy.

PHYSICIAN REVIEW

Two board-certified internists (D.Z.S. and M.D.A.) independently reviewed each case to determine whether the alert was valid on the basis of scientific data, published drug information, and clinical utility (ie, a good alert), and whether the prescribing physician's decision to override the alert was appropriate given the clinical scenario (ie, a good decision). For events that resulted in an injury to a patient, the reviewers determined whether an ADE was present and whether the ADE resulted from failure to observe a computer alert. Physician reviewers scored each item on a 4-point scale (definite, probable, possible, and unlikely). Differences were resolved by consensus. Interrater agreement regarding the validity of the alert was excellent (κ = 0.86) and regarding the appropriateness of the physicians' decision to override an alert was good (κ = 0.66).

DATA ANALYSIS

To evaluate the hypothesis that most physicians accept drug allergy and high-severity drug interaction alerts, we calculated the number and percentage of each type of alert physicians overrode. Next, using the medical chart review sample, we tabulated the drugs, allergies, and types of interaction that generated level 1 interaction or allergy-related computer alerts. We examined the relationship between the decision to prescribe and factors hypothesized to affect physicians' decisions. We included demographic factors, such as the age and sex of patient and physician (using years since graduation from medical school as a proxy for age in the latter group), and insurance type (as a proxy for patients' socioeconomic status). We included the number of medication allergies per patient, number of prescription medications, and number of medical problems present on the patient's problem list. We identified the level and type of physician practice (house officer, community-based physician, or hospital staff physician). We accounted for the type of alert (drug allergy or drug interaction), type of encounter (office visit or telephone contact), and new or renewed prescription.

We hypothesized that house officers would be more receptive to electronic alerts than more experienced physicians. We hypothesized that physicians would be less likely to override alerts in patients with multiple allergies, who were receiving many medications, or who had many comorbid medical illnesses. In these cases, physicians may have fewer available choices. We hypothesized that physicians would override more alerts for new prescriptions than for a changed dose or renewed prescription of a medication that the patient previously tolerated.

We used the χ2 statistic for comparisons involving categorical variables and the Fisher exact test to analyze ADE rates. We used a multivariable logistic regression model with forward selection (P<.20) to identify factors associated with physicians' decision to write a prescription that generated an alert. Analyses used Stata 7.0 software (Stata Corp, College Station, Tex).

SUMMARY OF ALERTS

During the 3-month study period, primary care physicians at the 5 study sites attempted to write 24 034 prescriptions using the OMR. These prescriptions generated 352 drug allergy and 3129 drug interaction alerts (after correcting for redundant keystrokes) for a total of 3481 alerts (Table 1). Primary care physicians overrode 91.2% of 352 drug allergy alerts, 89.4% of 574 level 1 drug interaction alerts, 96.3% of 2432 level 2 alerts, and 85.4% of 123 level 3 alerts. To understand factors that influenced prescribing behavior, we reviewed a sample of 189 alerts for detailed analysis.

Table Graphic Jump LocationTable 1. Drug Allergy and Drug Interaction Alerts in Primary Care, by Physicians' Decisions to Override
RESPONSIBLE DRUGS

In the medical chart review sample, 96 drugs generated 143 level 1 drug interaction alerts and 35 drugs generated 46 drug allergy alerts. The following 6 drugs accounted for approximately one third of alerts: cyclobenzaprine hydrochloride (16%), azithromycin (6%), atenolol (3%), a combination of trimethoprim and sulfamethoxazole (3%), clarithromycin (3%), and a combination of triamterene and hydrochlorthiazide (3%).

DRUG ALLERGY ALERTS

Antibiotics, cardiovascular medications, and analgesics accounted for 42 of 46 drug allergy alerts. Physicians overrode the analgesic allergy alert in 13 of 14 cases. In each case, the alert was triggered when a physician wrote a prescription for a drug that was in the same class as the drug to which the patient had an allergy. For example, hydrocodone bitartrate and oxycodone hydrochloride prescriptions triggered allergy alerts to codeine phosphate. Similarly, ibuprofen triggered an alert to naproxen.

In contrast, physicians overrode only 7 of 17 antibiotic allergy alerts. Prescriptions for amoxicillin and a combination of amoxicillin and clavulanate potassium triggered 6 penicillin allergy alerts; all but one was honored. In contrast, physicians overrode all 3 erythromycin allergy alerts triggered by prescriptions for azithromycin (all involved erythromycin intolerance rather than a true allergy). Overall, physicians overrode 31 of 46 drug allergy alerts included in the medical chart review.

DRUG INTERACTION ALERTS

Fifty-three different drug interactions triggered 143 drug interaction alerts. Four drug interactions accounted for 65 of 143 level 1 alerts. The sympathomimetic–tricyclic antidepressant interaction was triggered most often (43 of 143), followed by anticoagulant-macrolide and selective serotonin reuptake inhibitor–tricyclic interactions (8 of 143 for each), then triamterene- or amiloride hydrochloride–nonsteroidal anti-inflammatory drug interactions (6 of 143). Cyclobenzaprine accounted for more than half (24 of 43) of the sympathomimetic–tricyclic antidepressant interaction alerts and 6 of 8 selective serotonin reuptake inhibitor–tricyclic interaction alerts; azithromycin accounted for 6 of 8 anticoagulant-macrolide interaction alerts. Overall, physicians overrode 91 of 143 drug interaction alerts included in the medical chart review.

PATIENT AND PHYSICIAN ATTRIBUTES ASSOCIATED WITH PHYSICIANS' DECISIONS

We compared cases in which the primary care physician did and did not write a prescription for an alerted medication by factors hypothesized to affect physicians' decisions (Table 2). In the univariable analysis, the only factor associated with the decision to write a prescription for an alerted medication was prescription type. New prescriptions were less often written than renewals (50.0% vs 89.6% among alerted medications; P<.001).

Table Graphic Jump LocationTable 2. Attributes Associated With Prescription of an Alerted Medication*

In the multivariable logistic regression model (Table 3), physicians were less likely to prescribe (override) an alerted medication if the patient had multiple medication allergies (odds ratio [OR], 0.70; 95% confidence interval [CI], 0.53-0.93), and if the prescriber was a house officer (compared with a staff physician) (OR, 0.26; 95% CI, 0.08-0.84). Physicians were substantially more likely to override an alert for a renewal of a current prescription than for a new prescription (OR, 17.74; 95% CI, 5.60-56.18).

Table Graphic Jump LocationTable 3. Multivariable Analysis of Factors Associated With Physician Prescription of an Alerted Medication*
PHYSICIAN EXPLANATIONS

Thirty-two of 185 physicians in the study commented in the electronic medical record about their decision to observe or ignore a medication alert. In 15 of the 22 cases in which a physician had prescribed an alerted medication and commented about the decision, the physician wrote that the patient was not currently taking the medication listed as a potential cause of a level 1 interaction because it had been discontinued or the course had been completed. In 3 cases, the physician wrote that use of the prescribed medication was justified for a limited duration. In 2 more cases, the alert was prompted by a prescription of a medication with a listed allergy, but the drug was reported to be tolerated by the patient in the past. In the remaining 2 cases of allergy alerts, the physician indicated that the medication sheet or allergy information was incorrect.

REVIEWER ASSESSMENT OF ALERTS AND PHYSICIANS' DECISIONS

To assess independently the prescribers' decisions, 2 investigators reviewed each alert. Reviewers judged that 69 (36.5%) of 189 alerts were invalid (including 58 [40.6%] of 143 drug interaction and 11 [23.9%] of 46 drug allergy alerts). They judged that 37 (86.0%) of 43 sympathomimetic–tricyclic antidepressant interaction alerts, none of 8 anticoagulant-macrolide interaction alerts, none of 8 selective serotonin reuptake inhibitor–tricyclic interaction alerts, and none of 6 potassium-sparing diuretic–nonsteroidal anti-inflammatory drug interaction alerts were unjustified on the basis of scientific evidence.

Reviews also examined the prescribers' decisions to override an alert. Reviewers agreed with the prescribers' decisions in 185 (97.9%) of 189 cases, including 65 (95.6%) of 68 cases where the physician chose to override a valid alert. Many of these alerts provided worthwhile warnings. For example, most anticoagulant and antibiotic alerts were judged to be valid but appropriate to override (especially if a dose adjustment was recommended or follow-up laboratory tests were ordered). For each of the 119 cases in which a physician wrote a prescription that the reviewers judged appropriate, the reviewers indicated at least 1 reason to justify the decision (Table 4). Reviewers indicated most often that the patient was no longer taking the medication, the interaction was not clinically significant, the patient tolerated the drug(s), and the benefits of treatment outweighed the disadvantages.

Table Graphic Jump LocationTable 4. Appropriate Prescriptions for Alerted Medications: Reviewer Justifications (n = 119)
ADVERSE DRUG EVENTS

We identified 3 ADEs in the study, all affecting patients whose physicians had prescribed a drug that generated a level 1 drug interaction alert. One involved supratherapeutic anticoagulation (international normalized ratio, 8) in a patient receiving warfarin sodium for an aortic valve replacement and atrial fibrillation; the patient was prescribed an antibiotic (clarithromycin) for a lower respiratory tract infection, but had no bleeding complications. In another case, epistaxis developed that required an emergency department visit. The patient took warfarin for atrial fibrillation and was prescribed aspirin (325 mg/d) 3 weeks earlier for coronary artery disease. The international normalized ratio was in the therapeutic range, and the hematocrit level was stable. Reviewers judged that both ADEs were related to the alerted medications. In a third case, clonidine hydrochloride patch therapy was discontinued 1 month after it was started because the patient complained of itching at the site of the patch. The clonidine prescription had triggered an interaction alert with β-blockers; the ADE was judged unrelated to the interaction.

An ADE affected 3 (2.5%) of 122 patients whose physician had written a prescription for an alerted medication and none of the 67 patients whose physicians had observed the alert, a difference that did not reach statistical significance (P = .55). Reviewers judged only the first event (the warfarin-clarithromycin interaction) as an event that was potentially preventable had the prescribing physician adjusted the anticoagulant dosage in advance.

In this study of medication safety, we set out to examine whether primary care internists honor drug allergy and drug interaction alerts generated by a CPOE system, and to understand the factors that influenced physicians' behavior. Physicians overrode alerts in most cases.

This result is consistent with a study of alerts in the inpatient setting, in which physicians observed fewer than half of potentially life-threatening drug interaction alerts.6 However, clinicians' responses to medication alerts were less frequent than the responses reported in other studies of alerts and reminders in the inpatient setting. In one of the earliest studies, McDonald9 and other investigators1012 showed that physicians responded to 51% of computer-generated reminders about condition management and drug toxicities; 22% of physicians in the control group responded to similar conditions without the benefit of a reminder. The effect is also smaller than the compliance rates of 15% to 30% for preventive health reminders in ambulatory care.1315

To understand the factors that influenced physicians' decisions to write a prescription for an alerted medication, we sampled from alerts that physician had and had not overridden. We found that certain drugs (eg, cyclobenzaprine) and drug interactions (eg, tricyclic antidepressants and sympathomimetics) accounted for a disproportionate share of alerts. Physicians were more likely to override some alerts than others; this behavior may signal interactions that clinicians view with suspicion. In fact, physician reviewers judged that one third of alerts were inappropriate.

When we examined factors hypothesized to affect physicians' decision to prescribe an alerted medication, we found that physicians less often overrode alerts among patients with multiple drug allergies, suggesting that there were few therapeutic alternatives available. In addition, house officers were less likely than hospital-based faculty internists to override alerts. As novice physicians, house officers may be receptive to new information and to the introduction of technology into their practice. Rejection of alerts by community physicians and faculty physicians may reflect the skepticism of physicians with greater experience about some features of the CPOE system, such as out-of-date information, identification of interactions that were not clinically significant, failure to note patient tolerance of medication combinations, and the inability to balance the risks and benefits of therapy. It may also reflect a deeper-seated resistance among experienced practitioners to the perceived intrusion of information technology into the practice of clinical medicine.

Clinicians' acceptance of alerts and reminders reflects their relevance and validity. For example, Raschke and colleagues16 found that only 53% of 1116 medication alerts generated by a CPOE system at a 650-bed Arizona community teaching hospital accurately identified hazardous situations. A high volume of inappropriate or unhelpful alerts may discourage busy clinicians' willingness to examine, consider, and act on them.17,18

The Adverse Drug Event Prevention study by Bates et al1 showed that a sophisticated CPOE system can prevent medication errors among inpatients. Will CPOE prevent ADEs in primary care? We do not yet know which components of decision support in CPOE systems are necessary to produce improvements in primary care. Checking for drug allergy and interactions may be necessary but not sufficient to improve medication safety. Our results suggest that CPOE designers need to identify and eliminate inappropriate alerts that physicians find incredible, and change the threshold for generating alerts on renewals of medications that patients currently tolerate in combination. In addition, they should design advanced algorithms that ensure up-to-date and clinically relevant information is available to clinicians about laboratory data, dosing schedules, adverse drug reactions, and contraindications.1921

This study was subject to several limitations, including its small sample size and retrospective design. It was completed at 5 primary care practices affiliated with a single academic medical center using a single, shared CPOE system. The results may reflect in part features of the patients, clinicians, or information system peculiar to the study site and, hence, may not be generalizable to other practices and other systems. Our data suggest that ADEs occurred more often in cases where the physician failed to observe an alert, but the study was not adequately powered to address this question. We may have underestimated the number and type of ADEs because the medical chart review was limited to 3 months of follow-up. Since patients may fail to report or physicians to record ADE information in the medical record, more aggressive approaches to incident detection might also increase the yield of ADEs. In addition, the relatively short time frame of our study did not allow us to determine whether the override rate increases in a secular fashion. Physicians may increasingly override the system over time, if they perceive the alerts are usually inappropriate.

Nevertheless, the study offers information useful to the design of medication safety programs in primary care. Although a minority of primary care internists changed their prescription in response to an alert, most of these decisions were appropriate. Clinicians' skepticism about some CPOE medication alerts appears justified and indicates the importance of developing alert systems that are clinically useful and produce fewer unnecessary alerts. Specifically, systems should suppress alerts for renewals of previously tolerated medication combinations. Systems should prompt providers for information about the level of certainty associated with drug allergy alerts and to distinguish intolerance of well-recognized side effects from IgE-mediated responses. Systems should improve the signal-to-noise ratio by suppressing alerts with little evidentiary basis or clinical relevance and, in turn, provide access to references that can inform physician decisions.

In addition, systems should prompt providers to review medication lists for out-of-date information. They should also prompt clinicians to explain the basis for the decision to override an alert, so that this information can be used to inform the development of more effective decision support. Failure to document reasons for overrides may conceal sound clinical reasoning and create a liability risk for clinicians. In short, the development of more sophisticated decision support is needed to realize the promise of CPOE in primary care.

Corresponding author: Saul N. Weingart, MD, PhD, at the Division of General Medicine and Primary Care, Beth Israel Deaconess Medical Center, Rose 112, 330 Brookline Ave, Boston, MA 02215 (e-mail: sweingar@bidmc.harvard.edu).

Accepted for publication January 23, 2003.

This study was funded by a grant from the Stoneman Center for Quality Improvement in General Medicine and Primary Care, Beth Israel Deaconess Medical Center, Boston, Mass, and in part by a K08 Mentored Clinical Scientist Career Development Award (1 K08 HS11644-01) from the US Agency for Healthcare Research and Quality, Rockville, Md (Dr Weingart).

This study was presented as an abstract at the 25th Annual Meeting of the Society of General Internal Medicine; May 3, 2002; Atlanta, Ga.

We thank David W. Bates, MD, MSc, for his comments on an earlier draft of this article.

Bates  DWLeape  LLCullen  DJ  et al.  Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA. 1998;2801311- 1316
PubMed Link to Article
Massachusetts Coalition for the Prevention of Medical Errors, MHA best practice recommendations to reduce medication errors. Available at: http://www.macoalition.org/documents/Best_Practice_Medication_Errors.pdf. Accessed August 7, 2003.
Leapfrog Group, Fact sheet.  Washington, DC Academy Health June 2003Available at: http://www.leapfroggroup.org/FactSheets/LF_FactSheet.pdf. Accessed August 7, 2003.
Gandhi  TKWeingart  SNSeger  DS  et al.  Medication errors and adverse events in the ambulatory setting [abstract]. J Gen Intern Med. 2001;16 ((suppl)) 133
Gandhi  TKWeingart  SNBorus  J  et al.  Adverse drug events in ambulatory care. N Engl J Med. 2003;3481556- 1564
PubMed Link to Article
Peterson  JFKuperman  GShek  CBates  DW Physician responses to life-threatening drug-drug interaction alerts [abstract]. J Gen Intern Med. 2001;16 ((suppl)) 212
Tierney  WMMiller  MEOverhage  JMMcDonald  CJ Physician inpatient order writing on microcomputer workstations. JAMA. 1993;269379- 383
PubMed Link to Article
Safran  C Electronic medical records. JAMA. 2001;2851766
PubMed Link to Article
McDonald  CJ Protocol-based computer reminders: the quality of care and the non-perfectability of man. N Engl J Med. 1976;2951351- 1355
PubMed Link to Article
Kuperman  GJTeich  JMTanasijevic  MJ  et al.  Improving response to critical laboratory results with automation: results of a randomized controlled trial. J Am Med Inform Assoc. 1999;6512- 522
PubMed Link to Article
Kuperman  GJTeich  JMBates  DW  et al.  Detecting alerts, notifying the physician, and offering action items: a comprehensive alerting system. Proc AMIA Annu Fall Symp. 1996;704- 708
PubMed
McDonald  CJHui  SLSmith  DM  et al.  Reminders to physicians from an introspective computer medical record. Ann Intern Med. 1984;100130- 138
PubMed Link to Article
Tierney  WMHui  SLMcDonald  CJ Delayed feedback of physician performance versus immediate reminders to perform preventive care: effects on physician compliance. Med Care. 1986;24659- 666
PubMed Link to Article
McDonald  CJMurray  RJeris  DBhargava  BSeeger  JBlevins  L A computer-based record and clinical monitoring system for ambulatory care. Am J Public Health. 1977;67240- 245
PubMed Link to Article
McDonald  CJHui  SLTierney  WM Effects of computer reminders for influenza vaccination on morbidity during influenza epidemics. MD Comput. 1992;9304- 312
PubMed
Raschke  RABollihare  BWunderlich  TA  et al.  A computer alert system to prevent injury from adverse drug events. JAMA. 1998;2801317- 1320
PubMed Link to Article
Abookire  SATeich  JMSandige  H  et al.  Improving allergy alerting in a computerized physician order entry system. Proc AMIA Symp. 2000;2- 6
PubMed
Overhage  JMPerkins  STierney  WMMcDonald  CJ Controlled trial of direct physician order entry. J Am Med Inform Assoc. 2001;8361- 371
PubMed Link to Article
Rind  DMSafran  CPhillips  RS  et al.  Effect of computer-based alerts on the treatment and outcomes of hospitalized patients. Arch Intern Med. 1994;1541511- 1517
PubMed Link to Article
Kuperman  GJBates  DWTeich  JMSchneider  JRCheiman  D A new knowledge structure for drug-drug interactions. Proc Annu Symp Comput Appl Med Care. 1994;836- 840
PubMed
Schiff  GDAggarwal  HCKumar  SMcNutt  RA Prescribing potassium despite hyperkalemia. Am J Med. 2000;109494- 497
PubMed Link to Article

Figures

Place holder to copy figure label and caption
Figure 1.

Sample drug alert notification using the Online Medical Record. Hcl indicates hydrochloride; Celexa, citalopram hydrobromide; SSRI, selective serotonin reuptake inhibitor; trazodone, trazodone hydrochloride; and 5-HT, serotonin.

Graphic Jump Location
Place holder to copy figure label and caption
Figure 2.

Drug allergy and drug interaction alerts in primary care, by physicians' decisions to override. Hcl indicates hydrochloride; SSRI, selective serotonin reuptake inhibitor; trazodone, trazodone hydrochloride; and 5-HT, serotonin.

Graphic Jump Location

Tables

Table Graphic Jump LocationTable 1. Drug Allergy and Drug Interaction Alerts in Primary Care, by Physicians' Decisions to Override
Table Graphic Jump LocationTable 2. Attributes Associated With Prescription of an Alerted Medication*
Table Graphic Jump LocationTable 3. Multivariable Analysis of Factors Associated With Physician Prescription of an Alerted Medication*
Table Graphic Jump LocationTable 4. Appropriate Prescriptions for Alerted Medications: Reviewer Justifications (n = 119)

References

Bates  DWLeape  LLCullen  DJ  et al.  Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA. 1998;2801311- 1316
PubMed Link to Article
Massachusetts Coalition for the Prevention of Medical Errors, MHA best practice recommendations to reduce medication errors. Available at: http://www.macoalition.org/documents/Best_Practice_Medication_Errors.pdf. Accessed August 7, 2003.
Leapfrog Group, Fact sheet.  Washington, DC Academy Health June 2003Available at: http://www.leapfroggroup.org/FactSheets/LF_FactSheet.pdf. Accessed August 7, 2003.
Gandhi  TKWeingart  SNSeger  DS  et al.  Medication errors and adverse events in the ambulatory setting [abstract]. J Gen Intern Med. 2001;16 ((suppl)) 133
Gandhi  TKWeingart  SNBorus  J  et al.  Adverse drug events in ambulatory care. N Engl J Med. 2003;3481556- 1564
PubMed Link to Article
Peterson  JFKuperman  GShek  CBates  DW Physician responses to life-threatening drug-drug interaction alerts [abstract]. J Gen Intern Med. 2001;16 ((suppl)) 212
Tierney  WMMiller  MEOverhage  JMMcDonald  CJ Physician inpatient order writing on microcomputer workstations. JAMA. 1993;269379- 383
PubMed Link to Article
Safran  C Electronic medical records. JAMA. 2001;2851766
PubMed Link to Article
McDonald  CJ Protocol-based computer reminders: the quality of care and the non-perfectability of man. N Engl J Med. 1976;2951351- 1355
PubMed Link to Article
Kuperman  GJTeich  JMTanasijevic  MJ  et al.  Improving response to critical laboratory results with automation: results of a randomized controlled trial. J Am Med Inform Assoc. 1999;6512- 522
PubMed Link to Article
Kuperman  GJTeich  JMBates  DW  et al.  Detecting alerts, notifying the physician, and offering action items: a comprehensive alerting system. Proc AMIA Annu Fall Symp. 1996;704- 708
PubMed
McDonald  CJHui  SLSmith  DM  et al.  Reminders to physicians from an introspective computer medical record. Ann Intern Med. 1984;100130- 138
PubMed Link to Article
Tierney  WMHui  SLMcDonald  CJ Delayed feedback of physician performance versus immediate reminders to perform preventive care: effects on physician compliance. Med Care. 1986;24659- 666
PubMed Link to Article
McDonald  CJMurray  RJeris  DBhargava  BSeeger  JBlevins  L A computer-based record and clinical monitoring system for ambulatory care. Am J Public Health. 1977;67240- 245
PubMed Link to Article
McDonald  CJHui  SLTierney  WM Effects of computer reminders for influenza vaccination on morbidity during influenza epidemics. MD Comput. 1992;9304- 312
PubMed
Raschke  RABollihare  BWunderlich  TA  et al.  A computer alert system to prevent injury from adverse drug events. JAMA. 1998;2801317- 1320
PubMed Link to Article
Abookire  SATeich  JMSandige  H  et al.  Improving allergy alerting in a computerized physician order entry system. Proc AMIA Symp. 2000;2- 6
PubMed
Overhage  JMPerkins  STierney  WMMcDonald  CJ Controlled trial of direct physician order entry. J Am Med Inform Assoc. 2001;8361- 371
PubMed Link to Article
Rind  DMSafran  CPhillips  RS  et al.  Effect of computer-based alerts on the treatment and outcomes of hospitalized patients. Arch Intern Med. 1994;1541511- 1517
PubMed Link to Article
Kuperman  GJBates  DWTeich  JMSchneider  JRCheiman  D A new knowledge structure for drug-drug interactions. Proc Annu Symp Comput Appl Med Care. 1994;836- 840
PubMed
Schiff  GDAggarwal  HCKumar  SMcNutt  RA Prescribing potassium despite hyperkalemia. Am J Med. 2000;109494- 497
PubMed Link to Article

Correspondence

CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Please click the checkbox indicating that you have read the full article in order to submit your answers.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.
Submit a Comment

Multimedia

Some tools below are only available to our subscribers or users with an online account.

Web of Science® Times Cited: 196

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Collections
PubMed Articles
JAMAevidence.com

The Rational Clinical Examination
Cross-Reactivity With Other -Lactam Antibiotics