[Skip to Content]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
Purchase Options:
[Skip to Content Landing]
Views 3,907
Citations 0
April 12, 2018

Setting Achievable Benchmarks for Value-Based PaymentsNo Perfect Solution

Author Affiliations
  • 1The Joint Commission, Oakbrook, Illinois
JAMA. Published online April 12, 2018. doi:10.1001/jama.2018.2360

The goal of value-based purchasing is to reward clinicians and health care organizations that are able to achieve higher performance on quality measures. In principle, this is straightforward. But a challenge emerges when attempting to set a benchmark for payment. How high can quality get? For many measures, perfect (100% success or 0% failure) is theoretically possible. For example, it is possible that 100% of patients with coronary artery disease would either be prescribed antiplatelet therapy, have a valid contraindication, or make an informed choice to decline therapy.

However, in a test of the limits of quality improvement in clinical practice, 100% was not possible. Persell et al1 implemented a multifaceted intervention consisting of clinical decision support tools, documentation of medical contraindications or patient refusal, linked order sets, monthly notification of patients who were not taking “essential medications,” and audit and feedback with internal and external benchmarks. The study was conducted from 2007 to 2010 and included 1202 patients in an academic general internal medical clinic treated by 37 physicians. Over the first 12 months of the study, the rate of antiplatelet prescribing (or documentation of valid exception) for patients with coronary artery disease increased from 89% to 95%.1 Over the next 3 years, the rate improved further to approximately 97%. When the records of patients who did not satisfy the measure were reviewed, almost all had valid reasons for not meeting the quality indicator, such as recent treatment for gastric ulceration or pending surgery. Physicians had chosen not to use the clinical decision support system to document an exception because that would turn off the alert to remind them to restart antiplatelet therapy when the transient contraindication passed. Perfect performance was not possible because of imperfect performance measurement.

This example illustrates a general principle: as performance on a measure improves, the proportion of the quality gap (the difference between the measured performance and perfect performance) that is due to measurement error often increases. An example is The Joint Commission’s measure on early elective delivery (Perinatal Care Measure PC-01), which measures the proportion of deliveries with gestational age of 37 or more but less than 39 weeks that were elective vaginal deliveries (inductions) or elective cesarean births. When this measure was introduced in 2010, the mean and median rates of early elective delivery for the initial group of reporting hospitals (n = 167) were 17.9% and 16.1%, respectively. The Joint Commission’s Technical Expert Panel estimated that only 2% of valid exceptions were not captured, suggesting a large opportunity for improvement (written communication, Stephen Schmaltz, MS, MPH, PhD; biostatistician, Division of Healthcare Quality Evaluation, The Joint Commission; November 30, 2017). Over the next 6 years, the rate of early elective deliveries declined to 1.9% (n = 2055 hospitals).2 By 2016, 48% of reporting hospitals had a rate of 0% for the year; the median and 75th percentile rates were 0.6% and 3.0%, respectively, and 10% of hospitals had a rate of 5.7% or higher.2

What does the current very low rate of early elective deliveries mean for value-based purchasing? The Centers for Medicare & Medicaid Services (CMS) Hospital Inpatient Value Incentives and Quality Reporting program sets “thresholds” and “benchmarks” for payment formulas.3 The benchmark is the mean performance rate for the top decile of hospitals, and the threshold is the performance rate for the 50th percentile. Hospitals at or above the benchmark receive 10 points for the payment formula; those between the threshold and the benchmark receive 1 to 9 points; and those below the threshold receive no points.

For The Joint Commission’s early elective delivery measure (Perinatal Care Measure PC-01), the threshold set by CMS in 2017 was 3.125%, and the benchmark rate was 0%.4 Consequently, all hospitals with quarterly rates above 0% have a reduction in their quality score that could result in lower payments, even though many cases that cause a hospital’s rate to be slightly higher than 0% are probably due to medically appropriate care for unusual conditions not captured by the measure’s exceptions. A 2012 study of 107 145 deliveries at 108 Hospital Corporation of America–affiliated hospitals identified 205 (1.9%) cases that failed the PC-01 measure.5 Of the cases that failed, 71 (35% of early elective deliveries) had valid clinical indications that were not captured as exclusions. The denominators were small enough at these hospitals that a single false-positive failure on the measure would yield a PC-01 compliance rate of less than 95% in two-thirds of the hospitals in the study.

Over the last several years, The Joint Commission has received suggestions from the obstetrics community to adjust the specifications for the early elective delivery measure (PC-01) to allow for a wider array of exclusions so that hospitals do not have the potential for lower quality scores and lower payments from CMS that result from an early elective delivery that is actually clinically indicated. Consequently, some new International Classification of Diseases (ICD) codes were added to the exclusion list; others required the addition of new exclusions that can only be determined by chart review, which increases the burden of data collection and the cost of reporting the measure. For example, the data element “Prior Uterine Surgery” now includes a history or ultrasound documentation of thinning of the uterine wall and a history of a cornual ectopic pregnancy. The Joint Commission continues to receive numerous requests for “appeals” and new exclusions for uncommon or rare conditions to justify the need for an early-term elective delivery, almost all of which do not have a unique ICD code (eg, a mother with a malignancy who needed to start chemotherapy). While many conditions have been incorporated into the current PC-01 specifications, medical indications for early elective delivery are varied enough that it is impossible to enumerate 100% of the potential circumstances that could justify an early-term elective delivery.

Although it is unusual for a hospital to have a case for which an early elective delivery is clinically indicated but the reason is not captured as an exclusion, having a payment benchmark of 100% means that a single instance puts the hospital at risk for lower payments. This problem is exacerbated by calculating rates and payments quarterly, which reduces the denominator by 75% and multiplies the effect of a single failure by a factor of 4. For a hospital with 500 deliveries per year and a true performance rate of 0% on the early elective delivery measure (PC-01) over the year but with a single case with a valid exclusion that is not captured, the rate in the quarter in which the case occurred would be 0.8%, enough for a hospital to have a reduction of 2 or 3 points in the payment formula. Because of the payment benchmark, false positives (cases with valid indications for early elective delivery) on this measure could have an important effect on hospital payments.

National efforts have led to significant reductions in early elective deliveries, and the current gestational age distribution is more closely aligned to that seen in the generation prior to high rates of labor inductions and scheduled cesarean births. Although hospitals should continue to strive for 100% of early elective deliveries to have a valid clinical indication, performance on this measure should not be expected to reach 0%, nor should hospital payments in value-based purchasing programs be based on this benchmark. The Joint Commission supports a goal of zero patient harm, zero missed opportunities to deliver indicated clinical care, and zero unnecessary procedures but recognizes that may not always be possible.6 When measures are imperfect, as is usually the case, perfect performance should not be used for payment benchmarks. Instead, payers and policymakers need to understand the error inherent in a measure and take that into account when setting payment rules, especially when a high proportion of clinicians and hospitals have ideal performance rates.

Back to top
Article Information

Corresponding Author: David W. Baker, MD, MPH, The Joint Commission, One Renaissance Blvd, Oakbrook Terrance, IL 60181 (dbaker@jointcommission.org).

Published Online: April 12, 2018. doi:10.1001/jama.2018.2360

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest, and none were reported.

Persell  SD, Kaiser  D, Dolan  NC,  et al.  Changes in performance after implementation of a multifaceted electronic-health-record-based quality improvement system.  Med Care. 2011;49(2):117-125.PubMedGoogle ScholarCrossref
The Joint Commission. America’s hospitals: improving quality and safety—The Joint Commission’s 2017 annual report. http://jointcommission.new-media-release.com/2017_annual_report/. Accessed January 18, 2017.
Centers for Medicare & Medicaid Services. FY 2017 value-based purchasing domain weighting. https://www.qualityreportingcenter.com/wp-content/uploads/2015/02/IQR-FY2017_VBP-Domain-Weighting-Infographic.pdf. Accessed March 29, 2018.
Clark  SL, Meyers  JA, Milton  CG,  et al.  Validation of the joint commission exclusion criteria for elective early-term delivery.  Obstet Gynecol. 2014;123(1):29-33.PubMedGoogle ScholarCrossref
Dupree  ES. High reliability: the path to zero harm. https://www.centerfortransforminghealthcare.org/assets/4/9/ACHE_HC_Exec_article_Jan_2016.pdf. Accessed January 18, 2017.