The Electronic Medical Record:  A Randomized Trial of Its Impact on Primary Care Physicians' Initial Management of Major Depression FREE

MAJOR depression is among the most common problems encountered in primary care settings, affecting 6% to 10% of all patients who present for care.¹ Depressed patients experience at least as much physical and social dysfunction from this disorder as those with chronic physical conditions such as hypertension, diabetes, arthritis, and back pain.^2- 4 Moreover, depression worsens the prognosis for other coexisting medical problems and may even lead to suicide.^5- 6 In 1990 alone, the direct and indirect costs associated with depression in the United States exceeded $40 billion.⁷

Primary care physicians (PCPs) currently provide most treatment for depression.¹ However, poorer than expected outcomes are consistently reported for depressed patients treated by PCPs.^8- 9 This is thought to be partially related to PCPs' low rate of recognizing depression and a lack of awareness about and implementation of effective, guideline-based depression care.^10- 11 Screening primary care patients for depression, informing PCPs of the results, and then presenting patient-specific treatment recommendations should overcome these informational deficits. However, randomized clinical trials^12- 14 that have tested a screening and feedback strategy using a letter or note in the patient's medical chart addressed to the PCP have failed to demonstrate any significant improvement in clinical outcomes. These findings may have resulted from inadequate physician attention to the feedback process, physical separation between the written assessment and the patient's medical chart, an excessive interval between the PCP's contact with the patient and when he or she received notice of the psychiatric diagnosis, or failure by the physician to follow up and effectively treat the patient in a timely manner after receiving feedback.

Although the computer's potential to assist clinical decision making was recognized in the 1950s,¹⁵ recent advances in software and hardware technology, combined with reductions in the cost of computing power, have brought affordable electronic medical record (EMR) systems with decision support capabilities into the average physician's practice. At their most basic level, EMRs can facilitate the organization and rapid retrieval of information by serving as digital repositories for physicians' notes and laboratory reports as well as patients' problem lists, medications, allergies, and essential sociodemographic and contact data. Furthermore, some EMR systems are capable of presenting physicians with organized information in a timely manner and prompting them to provide appropriate clinical care.¹⁶ This strategy is particularly effective when the physician must specify an action taken in response to the notification and when initial nonresponders receive multiple messages.^17- 18 Indeed, physicians using EMR systems with these features are more likely to order recommended preventive and disease management care than practitioners using traditional, nonautomated medical records^19- 20 and have been found to make fewer prescription,²¹ test,²² and antibiotic ordering errors.²³ Routine use of EMR systems in clinical practice could also lead to a decrease in medical errors by eliminating illegible medical records and prescriptions. Furthermore, changes in clinical routines can be introduced into widespread practice through periodic software updates of existing protocols, thereby facilitating PCPs' ability to remain current and provide higher-quality care for a variety of conditions, including psychiatric disorders. Although it is estimated that just 5% of US physicians use computerized patient records in their office practices (David W. Bates, MD, Brigham and Women's Hospital, Boston, Mass, e-mail communication, May 30, 2000), increased applications may be anticipated given pressures to provide high-quality care at reduced cost, continued consolidation of health care providers into organizations sufficiently large to support the costs of installing and maintaining these systems, and steady declines in the real costs of computer technology.²⁴

Because PCPs are often ineffective at caring for patients with major depression, the EMR's technical features make it an attractive but untested vehicle for facilitating its treatment in routine primary care practice. Therefore, we investigated (1) how rapidly PCPs respond to an interactive message presented via a commercially available EMR system informing them that one of their patients had screened positive for major depression; (2) how often PCPs agree or disagree with the depression diagnosis when presented electronically; (3) clinical characteristics of patients and professional characteristics of PCPs associated with agreement with the diagnosis; and (4) PCPs' initial treatment recommendations after EMR feedback of the patient's depression diagnosis. Data regarding these practice patterns are highly pertinent for other investigators considering use of an EMR system to improve the quality of care for depression in the ambulatory medical sector.

This research was conducted at the main urban primary care practice affiliated with the University of Pittsburgh School of Medicine, Pittsburgh, Pa. This practice is staffed by 19 PCPs board certified in internal medicine who typically care for 10 to 15 patients per half day without assistance from house staff.

Nine months before implementing the electronic guideline for major depression, Logician (version 4.2; MedicaLogic, Beaverton, Ore)²⁵ was installed as the ambulatory EMR system for the practice. After each patient encounter, physicians type their clinical impressions directly into the EMR or dictate them into the medical center's central transcription service for later transcription and uploading into the EMR. In addition, physicians enter and maintain electronically their patients' problem and medication lists.

Primary care physicians can access their patients' medical information via computer terminals placed in examination rooms, common clinic work areas, or their own office located separate from the practice site. They use these terminals to obtain instant access to their patients' medical records. Primary care physicians are also given a printed summary of each patient's medical problems, medications, and various care recommendations on a paper encounter form generated by the EMR system for each office visit (Figure 1).

Using a protocol approved by the institutional review board of the University of Pittsburgh, we recruited PCPs approximately 1 month before patient recruitment. Several of us (B.L.R., W.N.K., and H.C.S.) presented highlights of the Agency for Healthcare Research and Quality (AHRQ) (formerly the Agency for Health Care Policy and Research) Depression Panel's Guideline²⁶ and then introduced the study to PCPs at an hour-long journal club–style conference. All 17 PCPs eligible to enroll in the study (B.L.R. and W.N.K. were ineligible to participate) subsequently provided informed consent to enroll and completed a self-report baseline assessment packet. Primary care physicians were then stratified by their number of half-day clinic sessions per week and then within each strata were randomly assigned to 1 of the 3 exposure conditions before the start of patient recruitment (see the "Intervention Conditions" subsection). Because of the nature of our interventions, PCPs were not masked to their assignment condition.

All patients aged 18 to 64 years presenting to the study site were screened for major depression using the self-administered Patient Questionnaire (PQ) portion of the Primary Care Evaluation of Mental Disorders (PRIME-MD),¹⁸ which they completed as part of routine practice before meeting with their PCP. If the patient screened positive for a mood disorder on the PQ and had (1) no obvious dementia, psychotic illness, or unstable medical condition; (2) 2 or fewer positive responses on the CAGE alcohol screening questionnaire²⁷ included on the PQ; and (3) no language or other communication barrier that would limit the patient's ability to participate in the research assessments, a research assistant sought the patient's written consent to administer the mood module component of the PRIME-MD to ascertain the presence of a Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition,²⁸ diagnosis of major depression.

When the PRIME-MD mood module assessment indicated that the patient had a current major depression, the research assistant asked the patient to provide a second written informed consent for the full clinical protocol. Each patient's clinical eligibility was confirmed with a detailed baseline telephone assessment and a medical chart review by one of us (B.L.R.). In addition to the criteria described in the previous paragraph, the protocol also required that the patient (1) have a Hamilton Depression Rating Scale score of 12 or greater, (2) report no alcohol or other substance abuse disorder within the past 2 months, (3) have no history of bipolar disorder, (4) have no active suicidal ideation, (5) be medically stable as determined from medical record review and the baseline telephone assessment, (6) have no plans to leave the study practice within the next 6 months, and (7) not be receiving treatment at baseline for depression from a mental health professional.

When a patient was identified by the mood module as having major depression, PCPs were notified via an interactive e-mail alert ("flag") generated through the EMR system and via an electronic letter signed by the study investigators. These messages generally were transmitted to the PCP within 1 business day of the patient being diagnosed as having major depression. Primary care physicians were asked to indicate whether they "agreed," "disagreed," or were "unsure" of the diagnosis of major depression on the PRIME-MD. They were also asked to electronically "sign" the letter to acknowledge its receipt, much as they would acknowledge a consult letter from another health care professional.

When the PCP indicated agreement with the psychiatric diagnosis, a researcher entered "major depression" into that patient's electronic problem list and forwarded a flag to the clinic's scheduling secretary. This message requested that the patient be scheduled or rescheduled for a follow-up visit with the PCP within 4 weeks of the PRIME-MD assessment, if such an appointment was not already entered into the practice's electronic scheduling system. If the PCP electronically expressed uncertainty about the depression diagnosis, the researcher replied with a new flag inquiring whether the patient could be scheduled to return within 4 weeks so that the PCP could again consider the diagnosis. When the PCP indicated disagreement with the depression diagnosis, another interactive e-mail message was sent after the patient's next visit. Reminders were automatically generated if the PCP did not respond to an e-mail message within 3 business days. The diagnosis of "major depression" was added to a patient's electronic problem list only when the PCP agreed with the PRIME-MD diagnosis.

"Usual care" clinicians and PCPs in the other treatment arms who disagreed or were unsure of a patient's depression diagnosis received no additional patient-specific treatment advice or reminders of care during follow-up.

"Passive care" PCPs were provided a reminder of their patients' depression diagnosis on the paper encounter form (Figure 1) generated for each patient visit. This message encouraged the PCP to treat the depressive episode but offered no details about how to do so. The message also suggested that the PCP mouse-click on a computer desktop icon if he or she wanted further advice for treating depression. Doing so launched a Web browser offering the PCP detailed advice for treating depression based on the AHRQ's depression treatment guideline²⁶ from an Intranet site we developed for use in this study. Passive care PCPs were not exposed to any other intervention prompts.

"Active care" PCPs who agreed with the diagnosis were exposed to 1 or more patient-specific advisory messages on the paper encounter form (Figure 1) generated for viewing at the time of the clinical encounter. These messages were based on the AHRQ practice guideline and were modified for electronic dissemination via the EMR system. Their content varied in keeping with a PCP's earlier actions as entered into the EMR system (eg, the patient was prescribed an antidepressant medication).²⁹ The clinician could also view these messages online at any time. Most messages concluded with a suggestion that the clinician mouse-click on the computer desktop icon to obtain further treatment advice from our Intranet site. Active care PCPs were also exposed to prompts offering to schedule a follow-up appointment with their study patients whenever the interval between follow-up appointments exceeded twice that recommended by the AHRQ guideline for a given treatment phase, as determined by a researcher's review of the clinicians' encounter notes.

All participants were contacted by telephone shortly after recruitment to confirm their protocol eligibility and to conduct a standardized research assessment. The interviewer (T.G.) was masked to the randomization status of a patient's PCP. These interviews assessed depression severity (using the Hamilton Depression Rating Scale),³⁰ quality of life (using the 12-Item Short-Form Health Survey),³¹ perceived social support (using the Medical Outcomes Study-social support scale),³² type of treatment received at the baseline visit for depression, and sociodemographic information. The type of depression-specific treatment recommended to the patient by the PCP and the number and timing of follow-up visits were abstracted directly from the EMR by another investigator (B.L.R.) who was also masked to the randomization status of a patient's PCP. This information was confirmed by patient self-report at the baseline assessment and during the 3-month follow-up telephone interview.

Shortly after protocol enrollment, PCPs completed questionnaires providing sociodemographic information, self-reported comfort using the EMR, and knowledge about and attitudes toward treating depression. Response time of PCPs to the electronic messages alerting them to their patients' PRIME-MD depression diagnoses and agreement with them were recorded by the investigator (T.G.) who had contacted the PCP and was masked to the randomization status of the patient's PCP.

The primary outcomes in this analysis of the EMR's utilization were (1) days to PCP response; (2) PCP initial response to notification of their patients' depression diagnosis; and (3) what actions the PCP took to treat the depressive episode. All statistical analyses were completed on an intent-to-treat basis using statistical software (Statistical Product and Service Solutions v9.0; SPSS Inc, Chicago, Ill, or Stata v6.0; Stata Corp, College Station, Tex).

Our sample size estimates were based on a 3-group design to detect a difference in proportion recovered of 30% (65% vs 35%), with α = .05 (2-tailed) and β = .20. Because the PCP was our unit of randomization and patients were nested under the PCP, we considered the dependence of observations and applied Donner's adjustment formula [1 + (n − 1)κ], where n is the number of patients nested under a physician and κ is the estimated intracluster dependence.³³ With a conservative κ of 0.05, we estimated before the start of the study that 72 patients were required in each group.

Patient variables across the 3 methods of exposing PCPs to guideline-based treatment advice were compared using χ² analyses that corrected for the nesting of patient under each physician. Comparisons among intervention arms in time to response to and time to agreement with the initial diagnosis flag were analyzed using discrete survival methods.³⁴ Univariate and multivariate correlates of the PCP's first response were identified by polychotomous logistic regression, which also corrected for the nesting of patients under each physician. The same statistical method was also used in comparing patients whose PCP agreed with the diagnosis within 3 days or 1 month of electronic notification or did not agree within 1 month. Physician actions after agreement (or disagreement) with the diagnosis were compared using separate logistic regressions.

The method used for regression modeling was to test for the significance of a variable first as a univariate correlate and then as a correlate, with intervention group added as an interaction term. Any variable that was a significant univariate correlate or whose interaction with intervention group was significant was included in a backward stepped multivariate model. Variables considered in the modeling included all patient variables portrayed in Table 1 and the PCP variables of years of experience, extent of depression training, number of clinic sessions per week, sex, attitudes toward treating depressed patients and the EMR system, and randomization status.

We approached 9513 patients aged 18 to 64 years between April 1997 and December 1998 and asked them to complete the PRIME-MD PQ (Figure 2). Of 8302 patients (87%) who did so, 1331 (16%) had positive screening results for a mood disorder (PQ+). After a research assistant performed a preliminary review of PQ+ patients' medical records, 736 (55%) were judged to be protocol eligible, 597 (81%) completed the mood module, and 345 (58%) met Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, criteria for a current episode of major depression. Of this group, 77 were protocol ineligible, 12 declined to enroll in the treatment phase, and we were unable to contact 8 patients to perform a baseline assessment and to confirm protocol eligibility despite multiple telephone calls and follow-up postcards. The remaining 227 patients were included in 1 of our 3 guideline exposure conditions in keeping with their PCP's earlier randomization assignment. Consequently, 78 patients were assigned to active care, 78 to passive care, and 71 to usual care. However, 15 patients (7%) later withdrew their consent to participate when contacted for a telephone assessment.

The analyses in this study are restricted to the 212 patients who completed the baseline telephone assessment. Measures of physician visits and mental health specialist use were available from 194 participants who completed a 3-month follow-up telephone interview. Table 1 presents demographic and clinical characteristics of study participants. Their mean age was 44 years (range, 19-64 years), 69% were women, and 72% were white. Although our 3 study groups were balanced on most demographic measures, active care patients were more likely to be men (P = .001) and passive care patients were more likely to be married (P = .04). There were small but statistically significant differences among the groups in the measures of depression severity, social support, and quality of life. The 3 groups were similar in their rates of a current comorbid anxiety disorder and past treatment for depression.

Although 17 PCPs were randomized into the 3 study arms, only 16 had patients enrolled in our study. One of the PCPs provided only demographic information. The other 15 completed most of the prestudy forms. The 3 study arms yielded physician groups that were similar in demographic and experience variables and in perceptions of depression care and the EMR (Table 2). Approximately half of the PCPs (8 of 15 respondents) reported some exposure to continuing medical education courses for treatment of depression, and three quarters (11 of 15) expressed comfort discussing depression care with their patients. Moreover, most study PCPs were comfortable using the EMR system for patient care.

Primary care physicians responded to the electronic flag notifying them of their patients' PRIME-MD depression diagnosis within 1 business day 54% of the time and within 3 business days 88% of the time. The median response time in the passive care condition was 2 days, but it was only 1 day for all groups combined (range, 1-95 days). Time to any response for active care physicians was 1.6 (95% confidence interval, 1.1-2.1) times faster than that for passive care PCPs. Time to any response for usual care physicians was intermediate and did not differ significantly from the other 2 groups.

Three days after notification, 120 (65%) of 186 PCP responses indicated agreement with the diagnosis of major depression generated by the PRIME-MD, 24 (13%) indicated disagreement, and 42 (23%) indicated uncertainty. Figure 3 displays the similarities in temporal pattern of agreement across study arms. At 3 days there were no statistically significant differences among PCP study arms in rate of agreement with the depression diagnosis. However, active care PCPs more frequently responded to the electronic flags than did PCPs in the other 2 groups (95% response rate vs 77% and 90% for passive care and usual care PCPs, respectively).

One month after notification, PCPs agreed with 147 (71%) of the depression diagnoses, disagreed with 34 (16%), and were unsure about 27 (13%). There were no significant differences in response rate or content of response at 1 month across intervention groups. Overall, PCPs responded to 208 (98%) of the electronic messages presented. The final PCP response was received on day 154 after electronic notification of the depression diagnosis. At this point, PCPs agreed with 166 (78%) of the depression diagnoses, disagreed with 36 (17%), and remained unsure about 10 (5%).

Multivariate analyses of PCP responses to the PRIME-MD diagnosis of major depression 3 days after notification are displayed in Table 3. Primary care physicians were less likely to disagree with the depression diagnosis if the patient had previously been treated for depression. However, PCPs with more than 10 years of clinical experience were more likely to disagree with the diagnosis. Female PCPs tended to be more unsure of the depression diagnosis than their male colleagues. Primary care physicians also ignored the initial EMR message regarding their patients' depression diagnoses less frequently if they were randomized to the active care study arm, they had more clinic sessions per week, or their patient was enrolled after the first 6 months of patient recruitment. When analyzed across all nonagreement responses, PCPs did not accept the depression diagnosis for patients with higher levels of baseline mental health quality of life (12-Item Short-Form Health Survey mental health composite score) and social support. Although PCP comfort with treating depressed patients and participation in continuing medical education for depression were associated with their response pattern, these variables correlated with having more years of clinical experience and were not significant in multivariate analyses. A model that included patient-level variables correctly predicted 56% of PCP responses (κ = 0.08; P<.01). Adding PCP-level variables to the predictive model provided little additional accuracy (correct predictions, 59%; κ = 0.18; P<.001).

When PCPs agreed with the depression diagnosis they typically documented depression in their progress notes and initiated treatment. Table 4 displays PCP actions by time to agreement with the diagnosis. Patients whose PCPs agreed with the diagnosis within 3 days received or were offered antidepressant pharmacotherapy significantly more often than those whose PCPs agreed later or did not agree at all. Patients in the 1-month agreement group were significantly more likely than the early agreement group to have a follow-up visit during this period, as recommended by the AHRQ Depression Panel's Guideline.²⁶ However, this pattern may reflect our attempts at having PCPs schedule an early follow-up appointment with patients if they were unsure about the depression diagnosis.

Primary care physicians responded quickly to electronic feedback of their patients' diagnosis of major depression on the PRIME-MD. Response rates and agreement with the diagnosis increased with electronic reminders and as study recruitment progressed. Primary care physicians who agreed with the depression diagnosis by day 3 after notification more quickly initiated antidepressant pharmacotherapy than PCPs who agreed with the diagnosis only at a later time. However, agreement with the diagnosis did not affect the rate at which patients were referred to a mental health specialist. Furthermore, PCPs initiated treatment for depression even when they disagreed or were unsure about the diagnosis, although at a lower rate than for patients about whose diagnosis they agreed. However, after controlling for agreement status, there were no significant differences by exposure of PCPs to intervention conditions in the initial treatments for depression offered to patients.

Investigators^12- 14 previously reported the results of screening primary care patients for major depression and then informing their clinicians of the diagnosis. However, this is the first study to our knowledge that informed the PCP of the depression diagnosis electronically or required the physician to acknowledge its receipt. Although PCP assignment to an AHRQ treatment guideline exposure condition did not affect level of diagnostic agreement (Figure 3), active care PCPs were less likely to ignore the electronic messages than were physicians in the other 2 groups. Primary care physicians were also less likely to ignore messages presented to them after the first 6 months of recruitment, suggesting a "learning curve" for the electronic feedback mechanism.

This study is also the first to analyze board-certified PCP rates of agreement with the PRIME-MD diagnosis of major depression. Although all study patients were experiencing at least a moderate level of depressive symptoms (Hamilton Depression Rating Scale score ≥12), PCPs disagreed with or were unsure about the depression diagnosis generated by the PRIME-MD for approximately one third of the patients 1 month after notification. Primary care physician agreement with the diagnosis 3 days after EMR notification was associated with poorer patient scores on the 12-Item Short-Form Health Survey mental health composite score but not the Hamilton Depression Rating Scale in the multivariate model (Table 3). Furthermore, experienced clinicians were more likely to rely on their own judgment and to disagree with the depression diagnosis generated by the PRIME-MD than were their less experienced colleagues.

It is tempting to speculate that electronic feedback of the depression diagnosis to PCPs can improve the speed with which the condition is recognized and appropriate treatment is begun. Indeed, higher rates of antidepressant pharmacotherapy were found in patients whose PCPs quickly agreed with the depressive diagnosis despite lack of a relation between diagnostic agreement and the severity of a patient's depressive episode. Thus, routine use of EMR systems may lead to pharmacotherapy being implemented at a guideline-recommended dosage and duration. This speculation is supported by findings from randomized trials in which EMRs increased the frequency with which recommended preventive and illness-specific care was provided^{17,19- 20,23,35- 43} and patient outcomes were improved.^{20,23,35,39,41- 42} Yet, we cannot confirm that electronic feedback of the diagnosis results in faster initiation of appropriate therapy because earlier studies of nonelectronic feedback of the diagnosis to PCPs did not describe the interval between feedback and the initiation of treatment. Moreover, screening and feedback of the depression diagnosis to PCPs via nonelectronic means has also been demonstrated to increase the recognition of mental disorders and the likelihood that an intervention will be initiated.^14- 16 Thus, further study of the EMR remains necessary to examine whether electronic feedback of the depression diagnosis and speedy initiation of treatment will increase PCP adherence to guideline-based treatment recommendations and improve clinical outcomes (eg, quality of life, work productivity, and reduced health services utilization).

Although agreement with the diagnosis of depression typically precedes treatment of the disorder, we found that PCPs sometimes prescribed an antidepressant drug for their patients even when disagreeing with the depression diagnosis (Table 4). We did not require PCPs to indicate why they disagreed with the diagnosis to minimize the protocol's intrusiveness. Still, we can speculate that some clinicians may be reluctant to diagnose major depression because it can offend patients.⁴⁴ More troubling is that PCPs often did not provide an antidepressant treatment, including a timely follow-up appointment, even when they agreed with the diagnosis.

The findings from this study must be interpreted with a recognition that it was conducted within a single, large, academically affiliated primary care practice. Although our main analyses corrected for the nesting of patient outcomes under each physician, our findings must be considered cautiously because each study group contained only 5 or 6 PCPs. This may have limited our power to detect significant differences among intervention groups or permitted the actions of a single PCP to unduly impact group outcomes. Also, investigators using other commercially available EMR systems may obtain different results. Given the trend toward rapid advancements in the capabilities of computer software, replication of this clinical trial using other contemporary EMR systems is necessary. Still, because EMRs cannot automatically capture the diagnosis of a mental illness as they can for an abnormal laboratory result or a drug-drug interaction, the need to systematically screen patients for depression in person,⁴⁵ by telephone⁴⁶ or after initiation of treatment⁴⁷ will remain for the foreseeable future.

We believe that our findings are generalizable to nonacademic group practice settings that use EMR systems in a similar manner as we have described. First, unlike many other studies conducted in academic settings, all of our study PCPs were board certified. House staff and their patients, including those precepted by attending physicians, were protocol ineligible. Second, our study site serves a diverse population that included individuals with lower to lower-middle income, residents of ethnic and African American neighborhoods, faculty and staff of the University of Pittsburgh, and others enrolled in a variety of insurance plans. Third, we know of no published study indicating that patients with psychiatric distress treated by "academic" PCPs achieve clinical outcomes dissimilar from patients treated by "nonacademic" PCPs.

In conclusion, electronic notification of the depression diagnosis can affect the PCP's initial management of major depression, particularly when the PCP agrees with the diagnosis. Further study is necessary to determine whether this strategy, combined with delivery of patient-specific guideline-based treatment recommendations, can improve adherence to an evidence-based treatment guideline and clinical outcomes for depression in routine primary care practice.

Accepted for publication July 11, 2000.

This work was supported by grant R01 HS09421 from the Agency for Healthcare Research and Quality, Rockville, Md.

Presented in part at the 13th NIMH International Conference on Mental Health Problems in the General Health Sector, Washington, DC, July 12, 1999.

Corresponding author: Bruce L. Rollman, MD, MPH, Center for Research on Health Care, University of Pittsburgh School of Medicine, 200 Lothrop St, Suite E-820, Pittsburgh, PA 15213-2582 (e-mail: rollmanbl@msx.upmc.edu).