Numbers of measures are listed according to measure collection and type of clinical service. A, Measure collections are grouped by sponsoring organization. Most measures (63%) were unique to one measure collection; the remainder were included in 2 to 7 collections (the sum of collection-specific measure totals therefore exceeds the total number of unique measures in the study). B, Sum of measure counts by service exceeds total number of measures (n = 521) because 90 measures addressed more than 1 clinical service. Measures current as of June 2012. CHIPRA indicates Children’s Health Insurance Program Reauthorization Act; CMS, Centers for Medicare & Medicaid Services.
aTotal depicted exceeds (by ≤2) the number of measures in the collection, due to measures with more than 1 target issue.
Newton EH, Zazzera EA, Van Moorsel G, Sirovich BE. Undermeasuring Overuse—An Examination of National Clinical Performance Measures. JAMA Intern Med. 2015;175(10):1709-1711. doi:10.1001/jamainternmed.2015.4025
Copyright 2015 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.
Clinical performance measures, designed both to evaluate and motivate clinician and institutional performance, have assumed a central role in efforts to improve the quality of US health care. Concerns have been raised, however, about the collective power of such measures to influence practice on a large scale.1,2 In particular, some worry that if measures predominantly target underuse of care—and incentives tend to reward clinicians for doing more—this could inadvertently contribute to the problem of overuse.3 We sought to determine whether and to what extent outpatient process measures preferentially target underuse compared with overuse.
We identified all outpatient and emergency department (ED) process measures contained in major national measure programs and clearinghouses (eg, National Quality Forum) as of mid-2012. We excluded measure collections addressing exclusively inpatients and collections of other measure collections, and excluded measures if they pertained to outcomes, settings other than outpatient office or ED, or nonclinical aspects of care. Across 16 measure collections, we identified 1037 unique measures, of which 521 (50.2%) met inclusion criteria. Institutional review board approval was not sought because no human subjects were involved.
Three coders independently categorized each measure according to target issue (underuse, overuse, or misuse; κ = 0.73) and clinical service addressed (eg, laboratory testing). Measures targeting underuse were defined as those asking “Has too little care been provided?”; overuse, “Has too much care been provided?”; and misuse, “Has care been provided incorrectly?” (Box). Coding rules were specified a priori. Coding discrepancies were reconciled by periodic consensus conference or, rarely, arbitrated by a fourth coder. We determined measure frequencies by target issue—overall, within each measure collection, and within each clinical service category. Analyses were performed using Stata statistical software (version 11.2; StataCorp).
Measures targeting underuse: “Has too little care been provided?”
The percentage of patients for whom a lipid panel is ordered within 3 months after being prescribed lipid-lowering medication (goal: high percentage)
The percentage of patients with deep vein thrombosis prescribed anticoagulation for at least 3 months after the diagnosis (goal: high percentage)
Measures targeting overuse: “Has too much care been provided?”
The percentage of patients undergoing back imaging within 28 days of a visit for new low back pain (goal: low percentage)
The percentage of patients dispensed an antibiotic within 3 days of diagnosis with bronchitis (goal: low percentage)
Measures targeting misuse: “Has care been provided incorrectly?”
The median time from emergency department arrival to time of transfer to another facility for acute coronary intervention (goal: shorter time)
The percentage of patients 18 years or older with pneumonia who receive their first dose of antibiotics within 6 hours after arrival at the hospital (goal: high percentage)
Of 521 unique measures that met inclusion criteria, 477 (91.6%) targeted underuse while 34 (6.5%) targeted overuse; 14 (2.7%) addressed misuse (4 measures addressed 2 target issues). Of 16 measure collections, just 3 contained an appreciable (≥10%) representation of overuse measures; nearly half (7 of 16) contained no overuse measures (Figure).
Most overuse measures (82.4%) addressed either diagnostic imaging or medication prescription (Figure). By comparison, underuse was well represented (over half of measures) as a target of measures across all categories of clinical service.
Providing high-quality health care requires both providing beneficial care and reducing nonbeneficial care. Increasingly, primary care clinicians3,4 and others2,5 worry that performance measurement may, through an emphasis on identifying and penalizing underuse, foster a culture of “more is better” and inadvertently encourage overuse. To our knowledge, our study is the first to systematically examine and quantify the existence of such an emphasis: current outpatient clinical process measures, both overall and within nearly all major measure collections, overwhelmingly target underuse of clinical services.
Performance measurement is well positioned to address both underuse and overuse—if, in the aggregate, clinicians are encouraged to ask themselves, “Am I doing enough for this patient without doing too much?” We believe our findings highlight the need to anticipate and monitor the aggregate effects—both intended and unintended—of measure program implementation. We would, moreover, advocate the development and implementation of a prospective underuse/overuse taxonomy as one means by which to promote greater balance across measure collections—or within individual measures—that simultaneously address underuse and overuse. Such a Goldilocks approach to performance measurement, as has been previously proposed,1,4 could encourage clinicians and institutions to target a balance of care that is just right.
Notwithstanding certain limitations of the present study, which focuses on measures themselves rather than on the putative connection between measure balance and physician behavior, we have shown that the current state of outpatient clinical performance measurement fails to address overuse—and may inadvertently reward it.
Accepted for Publication: June 17, 2015.
Corresponding Author: Erika H. Newton, MD, MPH, Department of Emergency Medicine, Stony Brook University Medical Center, HSC Level 4, Room 080, Stony Brook, NY 11794-8350 (email@example.com).
Published Online: August 10, 2015. doi:10.1001/jamainternmed.2015.4025.
Author Contributions: Dr Newton had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Newton, Sirovich.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Newton.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Newton, Sirovich.
Administrative, technical, or material support: Newton.
Study supervision: Sirovich.
Conflict of Interest Disclosures: None reported.
Disclaimer: The views expressed herein do not necessarily represent the views of the Department of Veterans Affairs or the US Government.