Patient-Reported Experiences of Dialysis Care Within a National Pay-for-Performance System

Key Points

Question  Are patient, dialysis facility, and geographic characteristics associated with dialysis facility performance on patient experience surveys?

Findings  In a cross-sectional analysis of 2939 US dialysis facilities, mean In-Center Hemodialysis Consumer Assessment of Healthcare Providers and Systems scores varied geographically and were lower in facilities owned by large chains as well as in for-profit and free-standing facilities. These scores were higher in facilities with more nurses per patient, and black and Native American populations reported less favorable experiences.

Meaning  The perceived quality of dialysis care delivered in certain settings appears to be of concern, and opportunities appear to exist for improved implementation of patient experience surveys in dialysis pay-for-performance programs.

Importance  Medicare’s End-Stage Renal Disease Quality Incentive Program incorporates measures of perceived value into reimbursement calculations. In 2016, patient experience became a clinical measure in the Quality Incentive Program scoring system. Dialysis facility performance in patient experience measures has not been studied at the national level to date.

Objective  To examine associations among dialysis facility performance with patient experience measures and patient, facility, and geographic characteristics.

Design  In this cross-sectional analysis, patients from a national end-stage renal disease registry receiving in-center hemodialysis in the United States on December 31, 2014, were linked with dialysis facility scores on the In-Center Hemodialysis Consumer Assessment of Healthcare Providers and Systems (ICH-CAHPS) survey. Of 4977 US dialysis facilities, 2939 (59.1%) reported ICH-CAHPS scores from April 8, 2015, through January 11, 2016. Multivariable linear regression models with geographic random effects were used to examine associations of facility ICH-CAHPS scores with patient, dialysis facility, and geographic characteristics and to identify the amount of total between-facility variation in patient experience scores explained by these categories. Data were analyzed from September 15, 2017, through June 1, 2018.

Exposures  Dialysis facility, geographic characteristic, and 10% change in patient characteristics.

Main Outcomes and Measures  Dialysis facility ICH-CAHPS scores and the total between-facility variation explained by different categories of characteristics.

Results  Of the 2939 facilities included in the analysis, adjusted mean ICH-CAHPS scores were 2.6 percentage points (95% CI, 1.5-3.7) lower in for-profit facilities, 1.6 percentage points (95% CI, 0.9-2.2) lower in facilities owned by large dialysis organizations, and 2.3 percentage points (95% CI, 0.5-4.2) lower in free-standing facilities compared with their counterparts. More nurses per patient was associated with 0.2 percentage points (95% CI, 0.03-0.3) higher scores; a privately insured patient population was associated with 1.2 percentage points (95% CI, 0.2-2.2) higher scores. Facilities with higher proportions of black patients had 0.95 percentage points (95% CI, 0.78-1.12) lower scores; more Native American patients, 1.00 percentage point (95% CI, 0.39-1.60) lower facility scores. Geographic location and dialysis facility characteristics explained larger proportions of the overall between-facility variation in ICH-CAHPS scores than did patient characteristics.

Conclusions and Relevance  This study suggests that for-profit operation, free-standing status, and large dialysis organization designation were associated with less favorable patient-reported experiences of care. Patient experience scores varied geographically, and black and Native American populations reported less favorable experiences. The study findings suggest that perceived quality of care delivered in these settings are of concern, and that there may be opportunities for improved implementation of patient experience surveys as is highlighted.

Landmark legislation in 1972 extended Medicare coverage to nearly all patients with end-stage renal disease (ESRD) and established what still is the only disease-specific federal entitlement program in the United States.¹ Owing to the federal government’s prominent role in financing ESRD care, the ESRD program serves as a bellwether for new federal payment initiatives. In 2012, the Centers for Medicare & Medicaid Services (CMS) enacted the ESRD Quality Incentive Program (QIP), the first legislatively mandated pay-for-performance program. The ESRD QIP incorporates facility performance on an array of quality measures when determining reimbursement.^2,3

As Medicare adopts more value-based payment models,^4-6 patient experience measures are becoming an important part of quality assessment initiatives.⁷ In 2002, CMS collaborated with the Agency for Healthcare Research and Quality (AHRQ) to develop the In-Center Hemodialysis Consumer Assessment of Healthcare Providers and Systems (ICH-CAHPS) survey, which became the first disease-specific patient experience survey used by CMS.^3,8-11 In 2016, ICH-CAHPS performance by dialysis facility was included as a clinical measure in the ESRD QIP, incorporating reports of patient experience into reimbursement calculations.¹²

A number of challenges exist to incorporating patient experience surveys in pay-for-performance programs, including the extent to which patient experiences measure an important dimension of the quality of care delivered,¹³ how to best administer surveys, and whether to apply risk adjustment to account for patient^14-17 and regional variation.^18-20 However, these surveys have the potential to introduce a patient-centered focus in quality assessment.⁷ Early experiences with ICH-CAHPS surveys can help policy refinement on diverse issues. In this study we examined the associations of dialysis facility ICH-CAHPS scores with patient, dialysis facility, and geographic characteristics.

We obtained ICH-CAHPS scores from surveys administered at US dialysis facilities from April 8, 2015, through January 11, 2016, and available on the Dialysis Facility Compare (DFC) website. Regulations require that third-party vendors administer ICH-CAHPS surveys. We obtained dialysis facility characteristics from facility surveys included in the United States Renal Data System (USRDS) database and from 2015 DFC reports. Facility characteristics included membership within a large dialysis organization (LDO), defined as a chain with more than 200 facilities. We identified characteristics of patients receiving dialysis at each facility on December 31, 2014, from the USRDS database (the most recent date for which USRDS data were available to us). We linked dialysis facility zip codes to US Census–based information on population density and the following 2 geographic units: hospital service areas and hospital referral regions (HRRs), where dialysis facilities are nested within hospital service areas and hospital service areas are nested within HRRs. This project was approved by the institutional review board of Baylor College of Medicine, Houston, Texas, which waived the need for informed consent for the use of publicly available databases.

Because most ICH-CAHPS scores are obtained from patients with Medicare, a population with readily available administrative claims data, we identified demographic, socioeconomic, and health characteristics among patients with at least 3 months of Medicare Parts A and B coverage before the end of 2014. We ascertained patient comorbidities from International Classification of Diseases, Ninth Revision, diagnosis codes listed on the prior 6 months of Medicare claims and from CMS Medical Evidence Reports (form CMS-2728), which nephrologists are instructed to complete for patients initiating maintenance dialysis. For each patient, we calculated a comorbidity index derived from a Charlson Comorbidity Index modified for ESRD (eMethods in the Supplement).^21,22 We merged data from the US Census to patient zip codes to approximate incomes.

We calculated a commonly used metric of market competition between dialysis facilities (the Hirschman-Herfindahl Index) from information about patients receiving dialysis at the end of 2014, the hospital service areas where they lived, and dialysis facility ownership. This index is associated with the number of competing dialysis facilities from which patients can choose.²³ We calculated this index for each dialysis facility using a previously described method applied to study competition in health care markets, including markets for dialysis care (eMethods in the Supplement).^23-25

The primary study outcome was the mean percentage of patients scoring their dialysis facility in the top box, or highest rating, across 6 patient experience and satisfaction categories, referred to as the mean ICH-CAHPS score. In this way, the ICH-CAHPS survey is incorporated into facility QIP scores. Three categories were composites of questions about nephrologists’ communication and care, dialysis center care and operations, and provision of information to patients, whereas 3 were overall ratings of satisfaction with nephrologists, dialysis center staff, and dialysis facilities (Box). We also examined facility scores on each of the 6 component categories.

Box Section Ref ID

Dialysis facilities with fewer than 30 adults receiving in-center hemodialysis are exempt from administering the ICH-CAHPS survey, whereas facilities that do not successfully administer 30 surveys are not given an ICH-CAHPS score. We examined the likelihood of qualifying facilities not meeting the threshold of 30 successfully completed surveys and refer to these as facilities with a missing ICH-CAHPS score. We also examined geographic variation in facilities’ mean ICH-CAHPS scores.

Our objective was to compare the associations of facility ICH-CAHPS scores with patient (10% difference), dialysis facility, and geographic characteristics listed in Table 1 and Table 2. We used HRRs to define each facility’s catchment area.

Data were analyzed from September 15, 2017, through June 1, 2018. We used multivariable mixed-effects linear regression models to examine the associations of facility ICH-CAHPS scores and patient, dialysis facility, and geographic characteristics. When examining the likelihood of a facility having a missing ICH-CAHPS score, we used logistic regression. In each regression model, we included the HRR where facilities were located as a random effect. All models included covariates listed in Table 1 and Table 2, with continuous covariates divided into categories to allow for nonlinear associations. We used a 2-tailed P < .05 as a measure of statistical significance.

To examine geographic variation in facilities’ ICH-CAHPS scores, we calculated the mean facility ICH-CAHPS scores within each HRR. We described variation in scores across HRRs and mapped quartiles of mean ICH-CAHPS scores across the United States. We also used a series of nested linear regression models to approximate the amount of total variance in facility ICH-CAHPS scores explained by patient characteristics, observed facility and geographic characteristics, and geographic location (eMethods in the Supplement).²⁶

Of 4977 US dialysis facilities with information from 2015 and early 2016 on the DFC website and identifiable characteristics, 2939 (59.1%) reported mean ICH-CAHPS scores. Among these facilities, the mean facility ICH-CAHPS score was 65.8% (SD, 8.3%; interquartile range, 60.3%-71.5%) (eFigure 2 in the Supplement). Facilities in the top 50th percentile of mean facility ICH-CAHPS scores cared for more white and Asian and older patients as well as more patients with private group health insurance, multiple medical comorbidities, less than 1 year of dialysis, and higher incomes. These higher-scoring facilities also cared for fewer patients who were black and who were eligible for Medicaid; they were smaller, had more nurses, technicians, and dialysis chairs per patient, and were in areas with less market competition (Table 1). Of the remaining 2038 (40.9%) facilities with missing ICH-CAHPS scores, 784 (38.5%) had fewer than 30 patients (eTable 2 in the Supplement).

Compared with nonprofit facilities, scores of for-profit facilities were lower by 4.2 percentage points; compared with hospital-based facilities, those of free-standing facilities were lower by 6.4 percentage points; and compared with independent or small-chain facilities, those of LDO facilities were lower by 2.6 percentage points (P < .001 for all comparisons). Compared with metropolitan facilities, scores were 2.3 percentage points higher for facilities in small towns, 1.4 percentage points higher for micropolitan facilities, and 0.4 percentage points lower for rural facilities (P = .002 for test of joint significance). Among facilities with nonmissing ICH-CAHPS scores, 2606 (88.7%) were for-profit, 2015 (68.6%) were owned by LDOs, 2825 (96.1%) were free-standing, and 104 (3.5%) were in small towns. Of all facilities, 1898 (64.6%) were owned by for-profit LDOs, all of which were free-standing (Table 2).

The multivariable regression model identified several patient characteristics that were independently associated with facility ICH-CAHPS scores. Mean ICH-CAHPS scores were lower by 0.95 percentage points (95% CI, 0.78 to 1.12) in facilities caring for higher proportions of black patient populations and by 1.00 percentage point (95% CI, 0.39 to 1.60) in facilities caring for larger Native American patient populations compared with facilities treating white patients. Facilities caring for higher proportions of privately insured patients had mean ICH-CAHPS scores higher by 1.17 percentage points (95% CI, 0.15-2.18) compared with facilities with fewer privately insured patients. Among facility characteristics examined, smaller size (fewer patients receiving dialysis) was associated with higher ICH-CAHPS scores; compared with facilities with more than 90 patients, those with fewer than 60 patients had mean scores higher by 3.01 percentage points (95% CI, 2.08-3.95). In addition, 1 more nurse per patient was associated with a higher mean score by 0.15 percentage points (95% CI, 0.02-0.28), whereas for-profit status (−2.62; 95% CI, −3.70 to −1.54), free-standing facilities (−2.25; 95% CI, −4.15 to −0.56), and LDO facilities (−1.58; 95% CI, −2.24 to −0.92) were associated with lower mean scores. Among geographic characteristics examined, location within a small town was associated with a higher mean ICH-CAHPS score of 1.87 percentage points (95% CI, 0.27-3.48) (Table 3 and Figure 1). Although having a larger proportion of patients in the third quartile of median income was associated with lower facility ICH-CAHPS scores, this finding did not persist in a sensitivity analysis (eMethods in the Supplement).

Among facilities with more than 30 patients, higher proportions of patients with multiple comorbidities (odds ratio [OR], 1.16; 95% CI, 1.04-1.31), immobility (OR, 1.30; 95% CI, 1.08-1.56), private insurance (OR, 1.78; 95% CI, 1.32-2.38), and Medicaid eligibility (OR, 1.11; 95% CI, 1.02-1.20) were associated with missing facility ICH-CAHPS scores. Caring for fewer patients was strongly associated with having a missing score (OR, 67.08; 95% CI, 44.66-100.75), and having more nurses (OR, 1.05; 95% CI, 1.01-1.09) and dialysis stations per patient (OR, 1.03; 95% CI, 1.02-1.04) were also associated with an increased likelihood of a missing score. Facilities that offered a night shift (OR, 0.72; 95% CI, 0.55-0.94) and those with larger proportions of older patients (OR, 0.83; 95% CI, 0.72-0.95) had decreased odds of a missing ICH-CAHPS score. For-profit and LDO status were not associated with missing surveys, whereas free-standing facilities were less likely to have a missing score compared with hospital-based facilities (OR, 0.30; 95% CI, 0.17-0.53) (eTable 3 in the Supplement).

The mean score among facilities in the top quartile of HRRs was 74%, compared with 61% among facilities in the bottom quartile of HRRs (P < .001) (Figure 2). Although facility ICH-CAHPS scores varied across HRRs, we did not observe clustering of ICH-CAHPS performance within broader geographic regions.

Our primary regression model explained 27% of the total observed variation in ICH-CAHPS scores among facilities. Facility locations (modeled as HRR random effects), patient characteristics, and observable facility and geographic characteristics independently accounted for 34%, 9%, and 18% of the total explained variation, respectively, whereas overlap among these 3 categories accounted for approximately 38% of the remaining explained variation. Separate models, including patient characteristics or facility locations alone, accounted for 45% and 60%, respectively, of the total explained variation (eTable 1 and eFigure 1 in the Supplement).

In companion multivariable models, predictors associated with the 6 individual components constituting the mean facility ICH-CAHPS score generally had the same direction of association as predictors associated with mean facility ICH-CAHPS scores, although not all predictors remained statistically significant. Several differences in the companion models were notable. A larger Hispanic population undergoing dialysis was associated with lower scores in the dialysis center care and operations composite (−0.26; 95% CI, −0.49 to −0.03), whereas larger Asian (−0.40; 95% CI, −0.68 to −0.11) and Hispanic (−0.46; 95% CI, −0.62 to −0.29) populations were associated with lower scores in the composite of providing information to patients and in the overall ratings of dialysis center staff. “Overall ratings of dialysis center staff” are different from the “composite of providing information to patients” (−0.43; 95% CI, −0.78 to −0.08, and −0.76; 95% CI, −1.36 to −0.16, respectively). Less market competition was associated with better scores on the providing information to patients composite (3.08; 95% CI, 1.85-4.31) as well as global ratings of facilities (OR, 2.64; 95% CI, 0.01-5.28) and center staff (OR, 2.66; 95% CI, 0.10-5.22) (eTables 4 and 5 in the Supplement).

We found that dialysis facility characteristics and geographic location were strongly associated with facility ICH-CAHPS scores. Mean scores for dialysis facilities that were for-profit or free-standing or that belonged to LDOs were lower by 1.6 to 2.6 percentage points, independently of other observable characteristics; based on these estimates, for-profit LDO facilities would be independently associated with mean scores lower by 4.2 percentage points. Facilities with more nurses per patient had better ICH-CAHPS performance. Among patient characteristics that were significantly associated with ICH-CAHPS scores, the proportion of patients of minority race/ethnicity had the largest estimated effects. Although 41.0% of facilities did not report ICH-CAHPS surveys, more than half of these facilities had the requisite number of patients to qualify for the performance measure had they successfully obtained 30 survey responses during the year.

Although patient experience scores are increasingly used to evaluate and compare health care providers, debate is ongoing about what constitutes a minimally important difference in scores. A recently published systematic review identified 3 general approaches that have been used to define a minimally important difference in group-level patient experience scores. These include comparing an observed score difference against the overall distribution of scores in the population, an external anchor known to be important to patients, and the differences observed in association with other important covariates.²⁷ By all of these criteria, the differences in ICH-CAHPS scores that we observed among different dialysis facility characteristics would be considered meaningful. For example, if a low-scoring dialysis facility were to improve its performance by 4.0 percentage points (or an SD of 0.5), it could rise 18 percentile points in its relative rank. Similar—or smaller—differences in patient experience scores were considered relevant in a comparison of hospitals and were found to correlate with patients’ disenrollment from health plans and choices of primary care physicians.^28-30 Further evaluating the extent to which patient-experience scores reported at the dialysis facility level reflect meaningful differences in care delivery will be increasingly important.

Our finding that reported mean patient experience scores are lower at facilities that are for-profit, free-standing, and part of an LDO is consistent with economic theory predicting that nonprofit health care providers place a greater emphasis on quality³¹ and with prior studies examining the associations among dialysis provider characteristics and other potential dimensions of care quality.^24,32 For example, mortality rates may be higher among patients undergoing dialysis at large dialysis chains,³² and patients undergoing dialysis at for-profit facilities spend more time in the hospital.²⁴ Although our findings raise concern that differences in care delivery may also contribute to lower reported experiences at for-profit, free-standing, and LDO facilities, the connection between patient experiences and the quality of care delivered by health care providers is also uncertain.^13,14,33-36

The role that geography plays in patient experience can inform efforts for risk adjustment of ICH-CAHPS survey scores. Ongoing efforts are examining inputs for equitably risk-adjusting quality measures.^37,38 Similar to prior analyses of quality in ESRD care that failed to detect differences in traditional quality measures across observed geographic characteristics,^39-41 we did not identify significant associations between facility ICH-CAHPS performance and most observed geographic characteristics. However, geographic location accounted for a substantial proportion of the total explained variation in ICH-CAHPS scores, suggesting that unobserved geographic factors, such as patients’ cultural and social preferences or regional differences in reimbursement incentives,^42,43 influence facility performance. To the extent that these unobserved geographic features are outside a dialysis facility’s control, this finding supports the use of geographic risk adjustment to “level the playing field” across US dialysis facilities. One approach to handle unobserved geographic differences in reporting patient experiences would be to compare dialysis facilities with other facilities operating in a similar region, as in some Medicare Shared Savings programs.⁴⁴

We identified several areas where the survey’s implementation within the ESRD QIP could be improved. Only 59.1% of US dialysis facilities had complete (or nonmissing) ICH-CAHPS scores. When analyzing factors associated with missing scores, we found that facilities that were smaller, cared for sicker populations, had more immobile patients, and had a higher proportion of patients requiring Medicaid coverage because of poverty or disability were significantly more likely to have missing survey responses, suggesting that specific patient populations may be less likely to respond to surveys. Although smaller facilities also had higher ICH-CAHPS scores, the sizeable number of smaller facilities with no score may have confounded this association. This bias would occur if the likelihood of having a missing score among smaller facilities is inversely associated with patient satisfaction. Future iterations of the ICH-CAHPS survey should aim to improve response rates in these facilities.

The observed associations of race and ethnicity with facility performance on ICH-CAHPS scores highlight an opportunity to improve patient experiences. We found that facilities with more black and Native American patients received lower scores on patient experience, consistent with evaluations of the CAHPS patient experience survey for home health services.⁴⁵ We also found that facilities with larger proportions of Hispanic and Asian patients (for whom English is often a nonprimary language) received poorer scores on facility communication. This variation in communication scores among racial and ethnic minorities is consistent with other (nondialysis) CAHPS surveys.⁴⁶ These findings underscore the potential influences of socioeconomics, cultural differences, and beliefs on experience reporting and highlight an important opportunity for dialysis providers to improve care delivered to racial and ethnic minorities.

This study included a national patient cohort. We leveraged comprehensive sources of patient, geographic, and facility information to account for many observed factors, including metrics of market competition. However, several limitations should be noted. We aggregated the characteristics of patients receiving dialysis from the most recent USRDS data available (December 2014) but used these characteristics to compare the scores reported from surveys administered in 2015 and early 2016. Although unlikely, we cannot exclude the possibility of meaningful changes in patient characteristics within facilities during 2015. The aggregate survey results from the DFC website did not include the number of individual survey respondents at each facility, which could disadvantage dialysis facilities with smaller patient populations. In addition, the DFC data did not allow us to determine predictors associated with individual patient scores, increasing the risk of erroneous associations resulting from the process of aggregation. When considering our results in the broader policy context, however, CMS uses facility-level data to calculate facility QIP scores, which aligns our vantage point with that of national policymakers. Certain patients may be more or less likely to complete the ICH-CAHPS survey, and survey administration methods may vary, which may lead to bias. We did not have information about self-reported health, which has been associated with ICH-CAHPS scores and is currently used by CMS in other health care settings to risk adjust CAHPS performance.⁴⁷ Finally, the method used to decompose variance in ICH-CAHPS scores has limitations, including the possibility of overfitting the data.²⁶

Our study represents an initial investigation of how patient, facility, and geographic characteristics are associated with dialysis facility performance on reporting of patient experience in the US ESRD pay-for-performance program. Our findings raise concern about perceived patient experiences at for-profit, free-standing dialysis facilities and LDOs and about the perceived quality of care provided to certain racial and ethnic populations. They suggest that staffing facilities with more nurses may lead to improvements in patient experience scores. This study also highlights the potential value of risk adjustment for unobserved geographic factors and the need for more effective survey administration among smaller facilities caring for elderly, at-risk populations. These results reveal an opportunity for evaluation-informed policy refinement in the evolution of Medicare payment reform.

Accepted for Publication: June 15, 2018.

Corresponding Author: Kevin F. Erickson, MD, MS, Selzman Institute for Kidney Health, Section of Nephrology, Baylor College of Medicine, 2002 Holcombe Blvd, Mail Code 152, Houston, TX 77030 (kevin.erickson@bcm.edu).

Published Online: September 10, 2018. doi:10.1001/jamainternmed.2018.3756

Author Contributions: Dr Erickson had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Brady, Zhao, Chertow, Erickson.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Brady, Zhao, Erickson.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Zhao, Niu, Erickson.

Obtained funding: Brady, Winkelmayer, Erickson.

Administrative, technical, or material support: Brady, Winkelmayer, Milstein, Chertow.

Supervision: Winkelmayer, Chertow, Erickson.

Conflict of Interest Disclosures: Dr Chertow reported serving on the board of directors of Satellite Healthcare. No other disclosures were reported.

Funding/Support: This study was supported by grant K23 1K23DK101693-01 from the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) (Dr Erickson). This material was also supported by grant CIN13-413 for the use of facilities and resources of the Houston Veterans Affairs Health Services Research and Development Center for Innovations in Quality, Effectiveness and Safety.

Role of the Sponsor/Funder: This work was conducted under a data use agreement between Dr Winkelmayer and the NIDDK, and an NIDDK officer reviewed the manuscript and approved it for submission. The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: The data reported herein have been supplied by the United States Renal Data System. The interpretation and reporting of these data are the responsibility of the authors and in no way should be seen as an official policy or interpretation of the US government. The opinions expressed are those of the authors and not necessarily those of the Department of Veterans Affairs, the US government or Baylor College of Medicine.