[Skip to Content]
[Skip to Content Landing]
Views 273
Invited Commentary
Physical Medicine and Rehabilitation
November 9, 2018

Statistical Quality Measures for Postacute Care Community DischargeThrough a Glass Darkly

Author Affiliations
  • 1Department of Health Management and Policy, University of Michigan School of Public Health, University of Michigan Medical School, Ann Arbor
  • 2Division of General Medicine, Department of Internal Medicine, University of Michigan Medical School, Ann Arbor
  • 3Department of Pediatrics and Communicable Diseases, University of Michigan Medical School, Ann Arbor
  • 4Center for Clinical Management Research, Veterns Affairs Ann Arbor Healthcare System, Ann Arbor, Michigan
JAMA Netw Open. 2018;1(7):e184303. doi:10.1001/jamanetworkopen.2018.4303

In their Original Investigation, Middleton et al1 report on a new “patient-centered national quality measure” that quantifies successful discharge to the community (ie, to home or self-care), defined as not experiencing unplanned rehospitalization or dying within the following 31 days, after inpatient rehabilitation among Medicare fee-for-service beneficiaries. This quality measure is part of the Improving Medicare Post–Acute Care Transformation (IMPACT) Act of 20142 and will be publicly reported for inpatient rehabilitation facilities in the United States as early as October 2018.3

This study1 identifies significant variation in this risk-adjusted outcome measure, suggesting that opportunities exist to improve efforts to further enhance the quality of postacute care while reducing institutional cost. Appropriately, they also wisely indicate the need for further research to better understand what facilitates successful community discharge and the need to validate this new measure as an indicator of high-quality care. However, with anticipated public reporting of this measure soon, its use in comparing inpatient rehabilitation facility performance will likely occur before additional research can validate its fidelity as a true quality measure. Unfortunately, this public reporting is also expected to be linked in the near future with rehabilitation facility payment changes, as it follows on the heels of other significant legislation affecting Medicare payment policies linked to measured variation in outcomes. These policies most recently have included the Hospital-Acquired Conditions Reduction Program, the Hospital Readmissions Reduction Program, and the Hospital Value-Based Purchasing Program, each stemming from the Affordable Care Act, and on the physician payment side the 2015 Medicare Access and CHIP Reauthorization Act (MACRA).4 Each of these congressionally mandated initiatives seek to move Medicare payment from volume- and episode-based to value-focused and population-based paradigms. In fact, in January 2015, the US Department of Health and Human Services announced its intent to tie 85% of all traditional Medicare payments to quality or value by 2016 and 90% of payments by 2018.

Virtually all of Medicare’s value-based purchasing programs have appeared to follow an 8-step formula for creating a statistical quality measure, for public reporting, and subsequently for reducing Medicare payments, including the measure studied by Middleton et al,1 which addressed steps 2 to 4. The 8 steps include the following:

  1. Identify a high-volume or high-cost clinical area; this first step and even the method of calculation is often mandated by Congress and detailed in the Federal Register.

  2. Show that variation exists in these defined outcomes in the population or among hospitals.

  3. Apply a risk adjustment to the population or hospital patient population. The risk adjustment can be an existing risk-adjustment system (eg, the Charlson Comorbidity Score), or one can develop a disease-specific risk adjuster using available data, often limited to administrative data.

  4. After risk adjustment, demonstrate that variation still exists in the putative quality measure in the population or among heath care facilities and physicians.

  5. Claim on face validity alone that there are outcomes that can be used to define good and bad care.

  6. Declare those populations’ health care facilities or physicians with the lowest rates of the outcomes of interest to be good providers and those with the highest rates to be bad providers.

  7. Apply payment rewards and/or sanctions to the lowest and highest cohorts of patients ranked statistically among hospitals and physicians in the distribution.

  8. Declare that payment has now been transformed to reflect differential value and quality.

This rapid transformation of Medicare payment involves measuring and comparing complex clinical care delivered by hospitals and other postacute care settings, usually using only administrative data, without accounting for other important factors outside the care delivered by the facilities that influence measured outcomes. These factors include social determinants of health, community resources, and family structures and should at the very least inject a note of caution into this process.

Recently, Barnett et al5 and Meddings et al6 examined the influence of these factors after applying the existing statistical risk adjusters to hospital readmissions as part of Medicare’s Hospital Readmission Reduction Program. Several patient factors not reliably available in administrative claims data have been identified as important influences on the likelihood of success after discharge, with failure measured as unplanned readmission within 30 days. These patient factors include, for example, patients’ functional limitations, cognitive impairment, and the use of assistive devices as well as social determinants of health such as income, social support, transportation, educational level, access to care, and postdischarge care community resources such as home health, skilled nursing facility, and access to follow-up appointments after discharge.

After correcting for the readmission risk models specific for each disease, Meddings et al6 then assessed the incremental improvement in our ability to predict hospital readmissions using survey-based information on each patient from the Health and Retirement Study. We demonstrated that these additional patient-level characteristics provided enhanced ability to predict hospital readmission beyond that provided by the existing and technically sophisticated risk-adjustment models. For example, for pneumonia readmissions, having 3 or more activities of daily living difficulties predicted an increased rate of hospital readmission, as did using home health care in the past 2 years. For patients with heart failure, having children and being in the highest quartile of income were associated with significantly fewer readmissions. These results highlight the fact that many covariates exist that are not captured in existing administrative data sets, yet have a significant influence on purported quality and value outcome measures.

History includes many examples of the rapid development and application of statistically based measures that highlight variability in practice processes that, on their face, seemed to identify plausible and potentially actionable quality issues. The pace of the adoption and implementation of these statistically based measures is quickening, despite some important early lessons learned regarding these types of measures. More than 30 years ago, there was a significant interest in so-called small-area variation.7,8 These studies highlighted differences among small geographic regions in use of services such as coronary angiography, carotid endarterectomy, upper gastrointestinal tract endoscopy, etc. One of the principal assumptions related to this demonstrated variation was that areas with significantly higher rates of use of medical services had an increased prevalence of inappropriate use (bad care), whereas those areas with lower rates were using services more parsimoniously (good care). A number of investigators associated with the RAND Corporation conducted studies, stratified by their differences in rates of use among these small geographic areas, and then applied clinical appropriateness criteria to determine whether or not those areas with significantly higher rates did indeed have higher rates of inappropriate use of the services. The investigators found this not to be the case. Although it seemed as if this appropriateness explanation would account for the higher rates, when subjected to careful analysis, the higher rates were determined not to be associated with disproportionately more inappropriate use of the service.

The move by Congress and Medicare to quickly transform payment from volume to value has necessitated the rapid adoption of the 8-step process to define statistical quality of care measures that can then be quickly linked to differential payment. A necessary task missing from this process is the research step confirming validity beyond simple face validity, in step 5. That is, after the risk-adjustment step and before the step that links payment to variation in the distribution of outcomes, one must validate that the reported differences in outcomes are linked directly to hospital- or institution-based processes. Without this necessary step, one cannot be certain that the variation is not associated with critical and unmeasured confounders such as a patient’s wealth, family support, availability of community resources, etc. In our rush to transform payment, we have created a payment paradigm without doing the work to demonstrate its alignment with hospital-controllable outcomes. We have created a system motivated by rising cost embracing the old adage, “we know the cost of everything but the value of nothing.” We must insist that the necessary step of validating that the extremes of the statistical distribution used to identify outliers for penalties truly represent different provider-actionable events. If the answer to the question “What could I have done to modify this bad outcome?” is “nothing,” then one might have identified random bad events or events outside of the control of the facility.

Back to top
Article Information

Published: November 9, 2018. doi:10.1001/jamanetworkopen.2018.4303

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2018 McMahon LF Jr et al. JAMA Network Open.

Corresponding Author: Laurence F. McMahon Jr, MD, MPH, Division of General Medicine, Department of Internal Medicine, University of Michigan Medical School, 2800 Plymouth Rd, Bldg 16, Room 427W, Ann Arbor, MI 48109 (lmcmahon@umich.edu).

Conflict of Interest Disclosures: Dr McMahon reported research funding by a grant from the Agency for Healthcare Research and Quality (AHRQ). Dr Meddings reported funding by AHRQ grants, contracts with the Health Research and Educational Trust funded by AHRQ and the Centers for Disease Control and Prevention, and funding from the Department of Veterans Affairs National Center for Patient Safety and the University of Michigan; receiving honoraria for lectures and teaching related to prevention and value-based policies involving catheter-associated urinary tract infection and hospital-acquired pressure ulcers; and a provisional patent on a device for improving the safety of inserting urinary catheters. No other disclosures were reported.

Disclaimer: The findings and conclusions in this article are those of the authors and do not necessarily represent the official position of the University of Michigan, the Department of Veterans Affairs, or AHRQ.

Middleton  A. Graham  JE, Bettger  JP, Haas  A, Ottenbacher  KO.  Facility and geographic variation in rates of successful community discharge after inpatient rehabilitation among Medicare fee-for-service beneficiaries.  JAMA Netw Open. 2018;1(7):e184332. doi:10.1001/jamanetworkopen.2018.4332Google Scholar
Improving Medicare Post–Acute Care Transformation Act of 2014. PL 113-185, 128 Stat 1953. https://www.govtrack.us/congress/bills/113/hr4994. Accessed October 3, 2018.
Centers For Medicare & Medicaid Services. Medicare issues fiscal year 2019 payment & policy changes for skilled nursing facilities. https://www.cms.gov/Newsroom/Fact-Sheets/Medicare-Issues-Fiscal-Year-2019-Payment-Policy-Changes-Skilled-Nursing-Facilities. July 31, 2018. Accessed September 17, 2018.
Congress.gov. 114th Congress. H.R.2 - Medicare Access and CHIP Reauthorization Act of 2015. https://www.congress.gov/bill/114th-congress/house-bill/2/text. Accessed October 3, 2018.
Barnett  ML, Hsu  J, McWilliams  JM.  Patient characteristics and differences in hospital readmission rates.  JAMA Intern Med. 2015;175(11):1803-1812. doi:10.1001/jamainternmed.2015.4660PubMedGoogle ScholarCrossref
Meddings  J, Reichert  H, Smith  SN,  et al.  The impact of disability and social determinants of health on condition-specific readmissions beyond Medicare risk adjustments: a cohort study.  J Gen Intern Med. 2017;32(1):71-80. doi:10.1007/s11606-016-3869-xPubMedGoogle ScholarCrossref
Chassin  MR, Kosecoff  J, Park  RE,  et al.  Does inappropriate use explain geographic variations in the use of health care services? a study of three procedures.  JAMA. 1987;258(18):2533-2537. doi:10.1001/jama.1987.03400180067028PubMedGoogle ScholarCrossref
Leape  LL, Park  RE, Solomon  DH, Chassin  MR, Kosecoff  J, Brook  RH.  Does inappropriate use explain small-area variations in the use of health care services?  JAMA. 1990;263(5):669-672. doi:10.1001/jama.1990.03440050063034PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Limit 140 characters
Limit 3600 characters or approximately 600 words