0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Research Letter |

Undermeasuring Overuse—An Examination of National Clinical Performance Measures FREE

Erika H. Newton, MD, MPH1; Erin A. Zazzera, RN, MPH2; Guillaume Van Moorsel, MPH, MLIS3; Brenda E. Sirovich, MD, MS4,5
[+] Author Affiliations
1Department of Emergency Medicine, Stony Brook University Medical Center, Stony Brook, New York
2Division of Trauma Surgery, Department of Surgery, Stony Brook University Medical Center, Stony Brook, New York
3Stamford Health System, Stamford, Connecticut
4Outcomes Group, VA Medical Center, White River Junction, Vermont
5The Dartmouth Institute for Health Policy and Clinical Practice, Hanover, New Hampshire
JAMA Intern Med. 2015;175(10):1709-1711. doi:10.1001/jamainternmed.2015.4025.
Text Size: A A A
Published online

Clinical performance measures, designed both to evaluate and motivate clinician and institutional performance, have assumed a central role in efforts to improve the quality of US health care. Concerns have been raised, however, about the collective power of such measures to influence practice on a large scale.1,2 In particular, some worry that if measures predominantly target underuse of care—and incentives tend to reward clinicians for doing more—this could inadvertently contribute to the problem of overuse.3 We sought to determine whether and to what extent outpatient process measures preferentially target underuse compared with overuse.

We identified all outpatient and emergency department (ED) process measures contained in major national measure programs and clearinghouses (eg, National Quality Forum) as of mid-2012. We excluded measure collections addressing exclusively inpatients and collections of other measure collections, and excluded measures if they pertained to outcomes, settings other than outpatient office or ED, or nonclinical aspects of care. Across 16 measure collections, we identified 1037 unique measures, of which 521 (50.2%) met inclusion criteria. Institutional review board approval was not sought because no human subjects were involved.

Three coders independently categorized each measure according to target issue (underuse, overuse, or misuse; κ = 0.73) and clinical service addressed (eg, laboratory testing). Measures targeting underuse were defined as those asking “Has too little care been provided?”; overuse, “Has too much care been provided?”; and misuse, “Has care been provided incorrectly?” (Box). Coding rules were specified a priori. Coding discrepancies were reconciled by periodic consensus conference or, rarely, arbitrated by a fourth coder. We determined measure frequencies by target issue—overall, within each measure collection, and within each clinical service category. Analyses were performed using Stata statistical software (version 11.2; StataCorp).

Box Section Ref ID

Box.
Examples of Clinical Performance Measures, by Target Issue
  • Measures targeting underuse: “Has too little care been provided?”

    • The percentage of patients for whom a lipid panel is ordered within 3 months after being prescribed lipid-lowering medication (goal: high percentage)

    • The percentage of patients with deep vein thrombosis prescribed anticoagulation for at least 3 months after the diagnosis (goal: high percentage)

  • Measures targeting overuse: “Has too much care been provided?”

    • The percentage of patients undergoing back imaging within 28 days of a visit for new low back pain (goal: low percentage)

    • The percentage of patients dispensed an antibiotic within 3 days of diagnosis with bronchitis (goal: low percentage)

  • Measures targeting misuse: “Has care been provided incorrectly?”

    • The median time from emergency department arrival to time of transfer to another facility for acute coronary intervention (goal: shorter time)

    • The percentage of patients 18 years or older with pneumonia who receive their first dose of antibiotics within 6 hours after arrival at the hospital (goal: high percentage)

Of 521 unique measures that met inclusion criteria, 477 (91.6%) targeted underuse while 34 (6.5%) targeted overuse; 14 (2.7%) addressed misuse (4 measures addressed 2 target issues). Of 16 measure collections, just 3 contained an appreciable (≥10%) representation of overuse measures; nearly half (7 of 16) contained no overuse measures (Figure).

Place holder to copy figure label and caption
Figure.
National Clinical Performance Measures Addressing Underuse vs Overuse vs Misuse of Health Care Services

Numbers of measures are listed according to measure collection and type of clinical service. A, Measure collections are grouped by sponsoring organization. Most measures (63%) were unique to one measure collection; the remainder were included in 2 to 7 collections (the sum of collection-specific measure totals therefore exceeds the total number of unique measures in the study). B, Sum of measure counts by service exceeds total number of measures (n = 521) because 90 measures addressed more than 1 clinical service. Measures current as of June 2012. CHIPRA indicates Children’s Health Insurance Program Reauthorization Act; CMS, Centers for Medicare & Medicaid Services.

aTotal depicted exceeds (by ≤2) the number of measures in the collection, due to measures with more than 1 target issue.

Graphic Jump Location

Most overuse measures (82.4%) addressed either diagnostic imaging or medication prescription (Figure). By comparison, underuse was well represented (over half of measures) as a target of measures across all categories of clinical service.

Providing high-quality health care requires both providing beneficial care and reducing nonbeneficial care. Increasingly, primary care clinicians3,4 and others2,5 worry that performance measurement may, through an emphasis on identifying and penalizing underuse, foster a culture of “more is better” and inadvertently encourage overuse. To our knowledge, our study is the first to systematically examine and quantify the existence of such an emphasis: current outpatient clinical process measures, both overall and within nearly all major measure collections, overwhelmingly target underuse of clinical services.

Performance measurement is well positioned to address both underuse and overuse—if, in the aggregate, clinicians are encouraged to ask themselves, “Am I doing enough for this patient without doing too much?” We believe our findings highlight the need to anticipate and monitor the aggregate effects—both intended and unintended—of measure program implementation. We would, moreover, advocate the development and implementation of a prospective underuse/overuse taxonomy as one means by which to promote greater balance across measure collections—or within individual measures—that simultaneously address underuse and overuse. Such a Goldilocks approach to performance measurement, as has been previously proposed,1,4 could encourage clinicians and institutions to target a balance of care that is just right.

Notwithstanding certain limitations of the present study, which focuses on measures themselves rather than on the putative connection between measure balance and physician behavior, we have shown that the current state of outpatient clinical performance measurement fails to address overuse—and may inadvertently reward it.

Accepted for Publication: June 17, 2015.

Corresponding Author: Erika H. Newton, MD, MPH, Department of Emergency Medicine, Stony Brook University Medical Center, HSC Level 4, Room 080, Stony Brook, NY 11794-8350 (erika.newton@stonybrookmedicine.edu).

Published Online: August 10, 2015. doi:10.1001/jamainternmed.2015.4025.

Author Contributions: Dr Newton had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Newton, Sirovich.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Newton.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Newton, Sirovich.

Administrative, technical, or material support: Newton.

Study supervision: Sirovich.

Conflict of Interest Disclosures: None reported.

Disclaimer: The views expressed herein do not necessarily represent the views of the Department of Veterans Affairs or the US Government.

Drozda  JP  Jr.  Physician performance measurement: the importance of understanding physician behavior. JAMA Intern Med. 2013;173(15):1444-1446.
PubMed   |  Link to Article
Kizer  KW, Kirsh  SR.  The double edged sword of performance measurement. J Gen Intern Med. 2012;27(4):395-397.
PubMed   |  Link to Article
Sirovich  BE, Woloshin  S, Schwartz  LM.  Too little? Too much? Primary care physicians’ views on US health care: a brief report. Arch Intern Med. 2011;171(17):1582-1585.
PubMed   |  Link to Article
Powell  AA, White  KM, Partin  MR,  et al.  Unintended consequences of implementing a national performance measurement system into local practice. J Gen Intern Med. 2012;27(4):405-412.
PubMed   |  Link to Article
Werner  RM, Asch  DA.  The unintended consequences of publicly reporting quality information. JAMA. 2005;293(10):1239-1244.
PubMed   |  Link to Article

Figures

Place holder to copy figure label and caption
Figure.
National Clinical Performance Measures Addressing Underuse vs Overuse vs Misuse of Health Care Services

Numbers of measures are listed according to measure collection and type of clinical service. A, Measure collections are grouped by sponsoring organization. Most measures (63%) were unique to one measure collection; the remainder were included in 2 to 7 collections (the sum of collection-specific measure totals therefore exceeds the total number of unique measures in the study). B, Sum of measure counts by service exceeds total number of measures (n = 521) because 90 measures addressed more than 1 clinical service. Measures current as of June 2012. CHIPRA indicates Children’s Health Insurance Program Reauthorization Act; CMS, Centers for Medicare & Medicaid Services.

aTotal depicted exceeds (by ≤2) the number of measures in the collection, due to measures with more than 1 target issue.

Graphic Jump Location

Tables

References

Drozda  JP  Jr.  Physician performance measurement: the importance of understanding physician behavior. JAMA Intern Med. 2013;173(15):1444-1446.
PubMed   |  Link to Article
Kizer  KW, Kirsh  SR.  The double edged sword of performance measurement. J Gen Intern Med. 2012;27(4):395-397.
PubMed   |  Link to Article
Sirovich  BE, Woloshin  S, Schwartz  LM.  Too little? Too much? Primary care physicians’ views on US health care: a brief report. Arch Intern Med. 2011;171(17):1582-1585.
PubMed   |  Link to Article
Powell  AA, White  KM, Partin  MR,  et al.  Unintended consequences of implementing a national performance measurement system into local practice. J Gen Intern Med. 2012;27(4):405-412.
PubMed   |  Link to Article
Werner  RM, Asch  DA.  The unintended consequences of publicly reporting quality information. JAMA. 2005;293(10):1239-1244.
PubMed   |  Link to Article

Correspondence

CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Please click the checkbox indicating that you have read the full article in order to submit your answers.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.

Multimedia

Some tools below are only available to our subscribers or users with an online account.

1,111 Views
1 Citations
×

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Collections
PubMed Articles
Jobs
JAMAevidence.com

Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice, 3rd ed
Clarifying Your Question

Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice, 3rd ed
Three Examples of Question Clarification