0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Research Letters |

Information Overload and Missed Test Results in Electronic Health Record–Based Settings FREE

Hardeep Singh, MD, MPH; Christiane Spitzmueller, PhD; Nancy J. Petersen, PhD; Mona K. Sawhney, MD, MS; Dean F. Sittig, PhD
[+] Author Affiliations

Author Affiliations: Houston VA Health Services Research and Development Center of Excellence, Center of Inquiry to Improve Outpatient Safety Through Effective Electronic Communication, Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas (Drs Singh, Petersen, and Sawhney); Section of Health Services Research, Department of Medicine, Baylor College of Medicine, Houston (Drs Singh, Petersen, and Sawhney); Department of Psychology, University of Houston, Houston (Dr Spitzmueller); and School of Biomedical Informatics and University of Texas–Memorial Hermann Center for Healthcare Quality and Safety, University of Texas Health Science Center at Houston (Dr Sittig).


JAMA Intern Med. 2013;173(8):702-704. doi:10.1001/2013.jamainternmed.61.
Text Size: A A A
Published online

Electronic health record (EHR)-based alerts are often used to notify practitioners of abnormal test results, but follow-up failures (missed results) continue to occur in outpatient settings.1 In the Department of Veterans Affairs (VA), abnormal test result alerts are generated automatically for prespecified abnormal laboratory values or manually by the interpreting radiologist when an unexpected finding is noted.24 Factors such as workflow, user behaviors, and organizational characteristics likely affect EHR-based test result follow-up.1,5 Thus, we examined the “sociotechnical” predictors of missed test results in the setting of EHR-based alerts.

From June 2010 through November 2010, we conducted a cross-sectional survey of VA primary care practitioners (PCPs); trainees and subspecialists were excluded.6 The survey content was informed by an 8-dimensional sociotechnical model7 representing multiple complex facets of EHR-based test result notification. Survey items assessed PCPs' perceptions of technological factors (eg, EHR notification software, its ease of use, content of alerts PCPs received, EHR user interface) and social factors (eg, workflow, people, and organizational policies and procedures) related to alert follow-up. We assessed potential information overload by asking if practitioners received more alerts than they could effectively manage or received too many alerts to focus on the most important ones.

After pilot testing, we administered the 105-item survey using a web-based survey administration service. To increase response rates, invitation e-mails and reminders were followed by telephone attempts to reach nonrespondents. In accordance with VA policies, we did not use monetary incentives for participation.

We defined primary outcomes related to missed test results based on respondents' answers to 2 items: outcome 1 (potential for missed results) based on “The alert notification system in Computerized Patient Record System (CPRS) as currently implemented makes it possible for practitioners to miss test results,” and outcome 2 (personal history of missed results) based on “In the past year, I missed abnormal lab or imaging test results that led to delayed patient care.”

We examined correlation coefficients to determine significant relationships between sociotechnical variables and each primary outcome. Variables significantly related to the outcomes in bivariate analyses were then included in multivariate regression analyses. For ease of interpretation, we subsequently recoded the outcome responses into dichotomous categories and conducted logistic regression analyses. Responses of “agree” or “strongly agree” were coded as affirmative. Independent variables in the logistic regression models were those that were significant for both outcomes in the initial linear models. Stepwise selection was used to identify predictors significantly related to the outcome (P ≤ .05).

Of 5001 PCPs invited, 2590 (51.8%) responded. The median number of alerts PCPs reported receiving each day was 63; 86.9% perceived the quantity of alerts they received to be excessive, and 69.6% reported receiving more alerts than they could effectively manage (marker of information overload).

Over half (55.6%) reported that the EHR notification system as currently implemented made it possible for practitioners to miss test results. Almost a third (29.8%) reported having personally missed results that led to care delays.

The Table indicates the significant predictors in logistic regression. Perceived ease of EHR use was related to a lower likelihood of both the perception of potentially missing results (odds ratio [OR], 0.52 [95% CI, 0.32-0.86]) and to reporting missed results that led to care delays (OR, 0.64 [95% CI, 0.43-0.96]). Greater concern over electronic hand-offs (ie, routing alerts to the EHR of a surrogate covering practitioner) was also related to potential for (OR, 2.00 [95% CI, 1.38-2.89]) and personal history of (OR, 1.86 [95% CI, 1.28-2.69]) missed test results. PCPs who reported receiving more alerts than was manageable (information overload) were more likely to report having missed results that led to delayed patient care (OR, 2.20 [95% CI, 1.37-3.52]). Notably, the number of alerts that respondents reported they received per day was unrelated to either outcome.

Table Graphic Jump LocationTable. Statistically Significant Predictors of Missed Test Results

Our data suggest that PCPs using comprehensive EHRs are vulnerable to information overload, which might lead them to miss important information. While the alert quantity data was self-reported, it was strikingly similar to what we found using objective methods querying the EHR of a single VA facility (mean, 56.4 per day per PCP).8

Our study also suggests an association between usability and patient safety in the context of missed results. Because the EHR and its user interface is the same across all VA facilities, these perceptions were likely affected by other sociotechnical contextual factors that affect EHR-based test result follow-up. Current efforts to measure and improve usability should encompass a broad, “real-world” context that includes broader sociotechnical facets of the work environment.7,9 Moreover, an isolated reduction in alert numbers without attention to the broader PCP experience related to information overload might be insufficient to improve outcomes.10

Because this was a cross-sectional survey, we cannot determine causation. Nevertheless, our findings suggest that missed results in EHRs might be related to information overload from alert notifications, electronic hand-offs in care, and practitioner perceptions of poor EHR usability. Interventions to improve safety of test result follow-up in EHRs must address these factors.

Correspondence: Dr Singh, Michael E. DeBakey Veterans Affairs Medical Center (152), 2002 Holcombe Blvd, Houston, TX 77030 (hardeeps@bcm.edu).

Published Online: March 4, 2013. doi:10.1001/2013.jamainternmed.61

Author Contributions: Dr Singh had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Singh, Spitzmueller, Sawhney, and Sittig. Acquisition of data: Singh and Spitzmueller. Analysis and interpretation of data: Singh, Spitzmueller, Petersen, Sawhney, and Sittig. Drafting of the manuscript: Singh, Spitzmueller, and Sittig. Critical revision of the manuscript for important intellectual content: Singh, Spitzmueller, Petersen, Sawhney, and Sittig. Statistical analysis: Spitzmueller and Petersen. Obtained funding: Singh. Administrative, technical, and material support: Singh and Sittig. Study supervision: Singh and Sittig.

Conflict of Interest Disclosures: None reported.

Funding/Support: The research reported herein was supported by the Department of Veterans Affairs National Center for Patient Safety, a National Institutes of Health K23 career development award (K23CA125585) to Dr Singh and in part by the Houston VA Health Services Research and Development Center of Excellence (HFP90-020). Dr Sittig is supported in part by a SHARP contract from the Office of the National Coordinator for Health Information Technology (ONC No. 10510592).

Additional Contributions: We thank additional members of our project team, Donna Espadas, BS; Daniel Murphy, MD, MBA; Michael Smith, PhD; and Archana Laxmisan, MD, MA (all affiliated with the Houston VA Health Services Research and Development Center of Excellence), for their valuable contributions to the survey. None of them received additional compensation for this work.

Disclaimer: The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs or any other funding agencies.

Callen JL, Westbrook JI, Georgiou A, Li J. Failure to follow-up test results for ambulatory patients: a systematic review [published online December 11, 2011].  J Gen Intern Med
PubMed  |  Link to Article
Singh H, Thomas EJ, Mani S,  et al.  Timely follow-up of abnormal diagnostic imaging test results in an outpatient setting: are electronic medical records achieving their potential?  Arch Intern Med. 2009;169(17):1578-1586
PubMed   |  Link to Article
Singh H, Thomas EJ, Sittig DF,  et al.  Notification of abnormal lab test results in an electronic medical record: do any safety concerns remain?  Am J Med. 2010;123(3):238-244
PubMed   |  Link to Article
Singh H, Vij MS. Eight recommendations for policies for communicating abnormal test results.  Jt Comm J Qual Patient Saf. 2010;36(5):226-232
PubMed
Hysong SJ, Sawhney MK, Wilson L,  et al.  Understanding the management of electronic test result notifications in the outpatient setting.  BMC Med Inform Decis Mak. 2011;11:22
PubMed   |  Link to Article
Singh H, Spitzmueller C, Petersen NJ,  et al.  Primary care practitioners' views on test result management in EHR-enabled health systems: a national survey [published online December 24, 2012].  J Am Med Inform AssocLink to Article
Sittig DF, Singh H. A new sociotechnical model for studying health information technology in complex adaptive healthcare systems.  Qual Saf Health Care. 2010;19:(suppl 3)  i68-i74
PubMed   |  Link to Article
Murphy DR, Reis B, Sittig DF, Singh H. Notifications received by primary care practitioners in electronic health records: a taxonomy and time analysis.  Am J Med. 2012;125(2):209- e1-e7
PubMed   |  Link to Article
Roth EM, Eggelston RG. Forging new evaluation paradigms: beyond statistical generalization. In: Patterson E, Miller J, eds. Macrocognition Metrics and Scenarios: Design and Evaluation for Real-World Teams. Burlington, VT: Ashgate Publishing; 2010:203-219
Singh H, Wilson L, Reis B, Sawhney MK, Espadas D, Sittig DF. Ten strategies to improve management of abnormal test result alerts in the electronic health record.  J Patient Saf. 2010;6(2):121-123
PubMed   |  Link to Article

Figures

Tables

Table Graphic Jump LocationTable. Statistically Significant Predictors of Missed Test Results

References

Callen JL, Westbrook JI, Georgiou A, Li J. Failure to follow-up test results for ambulatory patients: a systematic review [published online December 11, 2011].  J Gen Intern Med
PubMed  |  Link to Article
Singh H, Thomas EJ, Mani S,  et al.  Timely follow-up of abnormal diagnostic imaging test results in an outpatient setting: are electronic medical records achieving their potential?  Arch Intern Med. 2009;169(17):1578-1586
PubMed   |  Link to Article
Singh H, Thomas EJ, Sittig DF,  et al.  Notification of abnormal lab test results in an electronic medical record: do any safety concerns remain?  Am J Med. 2010;123(3):238-244
PubMed   |  Link to Article
Singh H, Vij MS. Eight recommendations for policies for communicating abnormal test results.  Jt Comm J Qual Patient Saf. 2010;36(5):226-232
PubMed
Hysong SJ, Sawhney MK, Wilson L,  et al.  Understanding the management of electronic test result notifications in the outpatient setting.  BMC Med Inform Decis Mak. 2011;11:22
PubMed   |  Link to Article
Singh H, Spitzmueller C, Petersen NJ,  et al.  Primary care practitioners' views on test result management in EHR-enabled health systems: a national survey [published online December 24, 2012].  J Am Med Inform AssocLink to Article
Sittig DF, Singh H. A new sociotechnical model for studying health information technology in complex adaptive healthcare systems.  Qual Saf Health Care. 2010;19:(suppl 3)  i68-i74
PubMed   |  Link to Article
Murphy DR, Reis B, Sittig DF, Singh H. Notifications received by primary care practitioners in electronic health records: a taxonomy and time analysis.  Am J Med. 2012;125(2):209- e1-e7
PubMed   |  Link to Article
Roth EM, Eggelston RG. Forging new evaluation paradigms: beyond statistical generalization. In: Patterson E, Miller J, eds. Macrocognition Metrics and Scenarios: Design and Evaluation for Real-World Teams. Burlington, VT: Ashgate Publishing; 2010:203-219
Singh H, Wilson L, Reis B, Sawhney MK, Espadas D, Sittig DF. Ten strategies to improve management of abnormal test result alerts in the electronic health record.  J Patient Saf. 2010;6(2):121-123
PubMed   |  Link to Article

Correspondence

CME
Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.
Submit a Comment

Multimedia

Some tools below are only available to our subscribers or users with an online account.

Web of Science® Times Cited: 4

Related Content

Customize your page view by dragging & repositioning the boxes below.

See Also...
Articles Related By Topic
Related Collections
PubMed Articles