Background Whether safety-net hospitals (SNHs) provide patient-centered care has important implications both for patient outcomes and for how these hospitals will fare under value-based purchasing (VBP). We sought to determine performance and improvement on measures of patient-reported hospital experience among SNHs compared with non-SNHs.
Methods Our sample consisted of 3096 US hospitals. We defined safety-net hospitals as those hospitals in the highest quartile of the Disproportionate Share Hospital (DSH) index, and we used national data on patient experience from the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey in 2007 and 2010 to examine overall hospital performance and improvement over time.
Results Safety-net hospitals had lower performance than non-SNHs on nearly all measures of patient experience. The greatest differences were in overall hospital rating, where patients in SNHs were less likely to rate the hospital a 9 or 10 on a 10-point scale compared with patients in non-SNHs (63.9% vs 69.5%; P < .001). Gaps were also sizeable for the proportion of patients who reported receiving discharge information (2.6 percentage point difference; P < .001) and who thought they always communicated well with physicians (2.2 percentage point difference; P < .001). Although both groups of hospitals improved from 2007 through 2010, the gap between SNHs and non-SNHs increased (3.8% in 2007 vs 5.6% in 2010; P = .08). Finally, SNHs had a 60% lower odds of meeting VBP performance benchmarks for hospital payments (odds ratio, 0.4; 95% CI, 0.3-0.5; P < .001) compared with non-SNHs.
Conclusions Safety-net hospitals have lower performance than non-SNHs on metrics of patient-reported experience, improved somewhat more slowly under public reporting, and are likely to fare poorly under VBP.
Patient-reported experience with health care is an essential measure of how well a health care system functions. As such, improving patient experience in hospitals has become a major priority for both clinical leaders and policy makers. Prior studies in multiple health care settings have shown that poor self-reported experiences with the health care system are associated with slower recovery from illness and a lower likelihood of adherence to prescribed treatment regimens.1-3 Consequently, suboptimal patient experience has important implications not only for the health of patients but also for health care costs, which increase when patients use more health care services because of poor recovery and nonadherence.4
Optimizing patient experience may be particularly important in safety-net hospitals (SNHs), which care for vulnerable and typically poor populations. These patients, on average, are sicker5 and tend to have lower levels of trust in the health care system.6,7 Therefore, they may be more susceptible to the negative effects of substandard experiences with health care providers. Beyond the importance of providing patient-centered care for this population, the changing fiscal landscape for hospital payments under the Patient Protection and Affordable Care Act8 has made the need to improve patient experience even more pressing. Value-based purchasing (VBP), a program run by the Centers for Medicare and Medicaid Services (CMS), now ties part of each hospital's payments to its performance on a set of quality measures.9 Under VBP, approximately 1% to 3% of total Medicare payments will be held back, and hospitals will receive some portion of that money based on how well they perform on the VBP metrics. A meaningful part of each hospital's performance score will be determined using metrics of patient-reported experience from the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey.10 For SNHs, ensuring high performance under VBP will be particularly critical to their economic viability.11 Yet, without robust data on how SNHs perform on measures of patient experience and how these hospitals have responded to 4 years of public reporting on these metrics, it is difficult to project how they are likely to fare under the new payment scheme.
Therefore, in this study, we used HCAHPS data to answer 4 questions: First, how well do SNHs perform on patient experience compared with non-SNHs? Second, are there certain subgroups of hospitals for which being an SNH is associated with particularly poor levels of performance? Third, over the 4 years during which patient experience has been publicly reported, do SNHs show any evidence of narrowing the gap with other hospitals? And finally, how do SNHs currently fare in relation to achievement thresholds in the VBP program? Understanding the degree to which being an SNH is associated with worse performance, independent of hospital characteristics that are largely out of the hospital's control (such as its size or location), is critically important to devising policies that ensure greater accountability while maintaining equity.
Our hospital sample was based on hospitals reporting HCAHPS and Disproportionate Share Hospital (DSH) index12 data to CMS in 2010. Of the 4742 acute-care hospitals in the United States, we excluded 1547 hospitals that did not report DSH index to CMS. Of these excluded hospitals, 1267 were small, rural critical access hospitals, which are paid using different formulas by CMS. Of the remaining 3195 hospitals, 99 did not report HCAHPS scores in 2010. Our sample included the remaining 3096 hospitals, which provide 91% of all the acute care for elderly Medicare fee-for-service beneficiaries in the US comparisons between hospitals that reported both HCAHPS scores and DSH index vs hospitals that reported neither HCAHPS scores nor DSH index (eTable 1).
The HCAHPS survey includes 27 questions regarding patients' experiences during hospitalization for a medical or surgical condition. Responses, scored as “never,” “sometimes,” “usually,” and “always,” are grouped into a series of composite quality measures: (1) communication with physicians, (2) communication with nurses, (3) communication about medications, (4) quality of nursing services, (5) presence of discharge planning, (6) pain management, (7) cleanliness, and (8) quietness of the hospital environment. In addition, there are 2 global measures of patient experience: an overall rating on a scale from 0 to 10, and an assessment of whether the patient would recommend the hospital to family and friends (definitely no, probably no, probably yes, and definitely yes). The data were adjusted for patient-level demographics and the survey administration mode.13 The VBP program will use scores on the 8 HCAHPS measures in their calculation of incentive payments—including the 6 composite domains (composite quality measures 1-6) and a combined measure of cleanliness and quietness of the hospital environment (composite quality measures 7 and 8)—and an overall rating of the hospital.9
We used the American Hospital Association survey14 to obtain hospital characteristics including size, ownership, location (region and urban vs rural), teaching status, nurse-staffing levels, percentage of patients with Medicaid insurance, percentage of patients with Medicare insurance, and presence or absence of a medical intensive care unit. We calculated the proportion of each hospital's discharged elderly patients who self-identified as being black from Medicare inpatient files. We used a standard measure of nurse-staffing levels by dividing the number of full-time–equivalent nurses on staff by 1000 patient-days.15-17 We obtained data on Medicare cost margins (net hospital income divided by total hospital revenue) to characterize hospitals' financial status.
Because there is no consensus on how to define an SNH, we used multiple approaches. Our primary approach, which has been widely used,18 defines hospitals in the highest quartile of DSH index as SNHs. The DSH index is reported by each hospital to CMS and is a combination of the proportion of elderly patients who are receiving Supplemental Security Income and the proportion of nonelderly patients who are receiving Medicaid, as represented in the following formula:
DSH Index = (Medicare Supplemental Security Income days/total Medicare days) + (Medicaid, non-Medicare days/ total patient days).
One advantage to the DSH index is that it identifies poor patients irrespective of age. Although the proportion of Medicaid patients can also be used to assess a hospital's safety-net status, this method fails to identify elderly patients who are poor because their hospital charges are covered by Medicare. Still, we performed sensitivity analyses using proportion Medicaid to determine safety-net status; the results were qualitatively similar and here we present only those results under the DSH index classification.
Our primary outcome was the patient-reported overall hospital rating. Because the 2 measures of overall experience (overall hospital rating, and whether the patient would recommend the hospital) are highly correlated15 and the VBP rules only include the overall rating as a performance metric, we used the overall rating as our primary outcome of interest. However, we also examined the proportion of patients who would definitely recommend the hospital. The results were qualitatively very similar and are not presented. Secondary outcomes included each of the individual composite measures (eg, communication with nurses, communication with physicians).
We first compared hospital and patient characteristics across DSH index quartiles using χ2 tests, analyses of variance, and Kruskal-Wallis tests, as appropriate.
To examine the relationship between HCAHPS performance and DSH index as a continuous variable, we used a nonparametric locally weighted bivariate Loess curve. We then divided the hospitals in our sample into DSH index quartiles for ease of presentation, treating DSH index quartile as a categorical variable. We assessed bivariate relationships between DSH index quartile and HCAHPS scores and then created multivariate generalized linear regression models including those variables that we believed were largely outside the hospitals' control, including the number of hospital beds, teaching status, profit status, location, census region, presence or absence of a medical intensive care unit, nurse-to-patient ratio, and the percentage of patients with Medicare as the source of payment. We repeated the analyses excluding nurse-to-patient ratio or proportion of Medicare patients and the results were qualitatively similar.
We hypothesized, based on prior literature,15,19 that the impact of safety-net status on patient experience might be less pronounced among larger, public, teaching hospitals, or institutions located in the Northeast. We serially tested for interactions and show subgroups in instances where the interactions were significant.
Next, given our interest in examining how SNHs have fared under 4 years of public reporting, we used regression models to examine the HCAHPS overall hospital rating in 2007 and 2010 and assessed whether the changes over time varied by DSH index quartile. On the basis of prior evidence that poor baseline performers improve the most under public reporting,20 we further adjusted for baseline (2007) performance. Finally, we examined the characteristics of SNHs that were associated with the greatest improvements over the 4-year period. All analyses were performed with DSH index quartile as an ordinal variable and repeated using DSH index as a continuous variable; the continuous results are available in eTable 2.
Finally, because VBP includes a “consistency” score on which hospitals score maximum points if they perform at or above the national median score for all 8 HCAHPS measures, we assessed the odds of meeting this benchmark for SNHs vs non-SNHs. We used bivariate instead of multivariate models because under VBP performance payment policies, there will be no adjustments made for hospital characteristics. We also calculated the odds of failing to score at or above the median on all of the 8 measures, which, under VBP, would correspond with a withholding of the entire incentive payment allotted to a hospital.
Statistical analyses were performed using Stata software (version 12; StataCorp). The project was considered exempt by the human subjects committee of the Harvard School of Public Health.
Of the 3096 hospitals included in our analyses, 769 were in the highest DSH index quartile and thus composed the SNH group (Table 1). Safety-net hospitals were more likely than non-SNHs to be large hospitals that were for profit or publicly owned. Safety-net hospitals were more likely to be major teaching hospitals (18% in SNHs vs 4% among hospitals in the lowest DSH index quartile; P < .001) and were more often located in the South (52% vs 31%; P < .001) and West (27% vs 16%; P < .001). Safety-net hospitals had substantially lower nurses per 1000 patient-days than hospitals in the lower DSH index quartiles (5.8 vs 6.8; P < .001). Safety-net hospitals had fewer Medicare patients and more Medicaid and black patients than other hospitals. When we examined Medicare cost margins, we found that hospitals in the highest DSH index quartile had about 40% lower margins (P < .001 for difference across quartiles) than hospitals in the lowest DSH index quartile (Table 1).
Safety-net status and hcahps scores
When we examined the relationship between DSH index as a continuous variable and HCAHPS scores, we found that higher DSH index was associated with worse performance on overall hospital rating (Figure 1).
When the DSH index was examined in quartiles, SNHs had the lowest HCAHPS scores on all measures of patient experience but one (quietness of hospital environment; Table 2). The greatest differences in scores between SNHs and hospitals in the lowest DSH index quartile were on measures of the overall hospital rating (63.9% vs 69.5%, 5.6 percentage point difference; P < .001) and receipt of discharge information (2.6 percentage point difference; P < .001). Analyses treating DSH index as a continuous variable were similar (eTable 3).
Safety-net status and hcahps scores by subgroups of hospitals
When we tested for effect modification, we found that only the interaction terms for hospital size and region were statistically significant. We found modest differences in performance on the overall rating between SNHs and non-SNHs based on size (Table 3) but larger differences based on region. However, contrary to our hypothesis, gaps between SNHs and non-SNHs were smallest in the South (difference of 0.6% in overall rating) and largest in the Northeast (difference of 6.0 percentage points; P < .001 for interaction).
Changes in hcahps scores under public reporting
Safety-net hospitals had worse performance on overall hospital rating than non-SNHs in 2007 (the beginning of the public reporting period). Over the subsequent 3 years, all groups of hospitals improved, but the improvement was somewhat smaller among SNHs compared with non-SNHs (3.0% vs 4.8% among the lowest DSH index quartile, although the P value for difference was nonsignificant [P = .08]; Figure 2), leading to a wider performance gap in 2010 (5.6%) than in 2007 (3.8%). Analyses accounting for baseline performance and analyses treating DSH index as a continuous variable were similar (eTable 3a and eTable 3b).
When we examined which SNHs improved more (vs less) than the median, we found that characteristics associated with greater improvement included being larger, being for profit (and not being a public hospital), being a teaching hospital, and serving a smaller proportion of black patients (eTable 4).
SNHs AND VBP ACHIEVEMENT THRESHOLDS
We found that in 2010, while 26% of hospitals in the lowest DSH index quartile scored at or above the median on all 8 HCAHPS measures of patient experience (a key metric under VBP), only 11% of SNHs did so (Table 4). Safety-net hospitals had a 60% lower odds of achieving this benchmark (odds ratio, 0.4; 95% CI, 0.3-0.5; P < .001) than hospitals in the lowest DSH index quartile. Similarly, SNHs were far more likely to fail to score at or above the median on any of the 8 HCAHPS measures (Table 4). The results from multivariate models were similar (eTable 5).
In this national sample of US hospitals, we found that SNHs received lower ratings on measures of patient experience than non-SNHs. The gaps in performance between hospitals in the highest and lowest quartiles of DSH index were sizeable and are increasing over time. These differences, which varied depending on the type of SNH, translated into a striking disparity in the odds of scoring at or above the median on all 8 HCAHPS measures used in pay-for-performance policies, suggesting that these hospitals may fare poorly under the VBP payment scheme.
Our findings have important implications for the health and health outcomes of patients seeking care in these hospitals. Prior studies have shown that hospitals with the highest HCAHPS scores perform approximately 2 to 4 percentage points higher on measures for completion of clinical processes than hospitals with the lowest HCAHPS scores,15 and the differences we observed based on safety-net status were much larger than differences based on any other hospital characteristics we could identify. Patients who receive care in SNHs tend to be of lower socioeconomic status and are more often underrepresented minorities.21 These groups are at higher risk of nonadherence to treatment regimens,22-24 experience delays in care and longer hospital stays,25 and have been reported to have lower levels of trust in the health care system.6,7 Given their risk profile, these vulnerable patients would likely benefit the most from high-quality patient-centered experience, which has been shown to be associated with several measures of improved health status in many care settings.26,27 Despite the greater need to deliver patient-centered care for vulnerable populations at SNHs, our findings suggest that these hospitals are failing to do so.
The lower performance of SNHs on HCAHPS measures may have meaningful consequences under VBP, particularly since many SNHs operate under very small financial margins.28 While tying payments to quality metrics should promote better care, whether these financially constrained hospitals will be able to respond effectively is unclear.29 There is some evidence that under financial incentives, SNHs are able to improve their performance on processes of care, such as prescribing aspirin to patients with acute myocardial infarction.30 However, improving patients' experience may be substantially more challenging. Although the new VBP rules pay for improvements as well as achievement, our findings suggest that SNHs face challenges on both fronts. If SNHs are unable to substantially improve patients' experiences over the next several years, hospital-based incentive programs are likely to disproportionately penalize these institutions.
While prior studies have demonstrated that SNHs perform poorly on quality metrics based on processes of care,31-33 to our knowledge, there are no prior studies examining patient experience in SNHs among a nationally representative group of institutions. Although an earlier study showed that hospitals with a higher proportion of Medicaid patients tend to have lower performance on HCAHPS,34 this was at a time when nearly half of hospitals had chosen not to publicly disclose their performance. Our study examines data over a 4-year period and uses a larger, more representative sample of US hospitals.
Our study has limitations. First, while we chose a priori to define “safety-net hospitals” according to DSH index quartiles, this is an imperfect measure.35,36 Nonetheless, we found a consistent relationship between safety-net status and HCAHPS performance even when we used other approaches to identifying SNHs, which was reassuring. Second, we were only able to examine data over a 4-year period and it is possible that hospital efforts to improve their performance may take longer. Tracking how this group of hospitals fares over time will be critically important.
Another limitation is that HCAHPS measures are, by their nature, subjective. Therefore, we cannot differentiate between actual differences in care delivery vs differences in expectations of the underlying patient population. However, we are unaware of any data that suggest that poor patients have higher expectations; we suspect they may actually have lower expectations of service quality. If so, we may be underestimating the true gap in performance between SNHs and other hospitals. The HCAHPS measures are adjusted for patient self-reported health status and survey mode,13 which helps mitigate differences in performance based on variations in the patient population. However, other factors such as case-mix and race may still influence hospital performance on these measures.
In conclusion, we found that US SNHs performed more poorly than other hospitals on nearly every measure of patient experience and that gaps in performance were sizeable and persistent over time. These findings have important consequences for patients who receive care at these institutions and should renew our focus on helping these hospitals improve. Given that hospital payments are now tied to performance on these measures, we need renewed efforts to track performance of SNHs under VBP and may need specific quality-improvement programs targeting these organizations. Safety-net hospitals play a critical role in providing medical care to vulnerable populations, and ensuring that efforts to improve the quality of care at US hospitals do not worsen existing disparities will be a key challenge to policy makers.
Correspondence: Ashish K. Jha, MD, MPH, Department of Health Policy and Management, Harvard Scholl of Public Health, 677 Huntington Ave, Boston, MA 02115 (ajha@hsph.harvard.edu).
Accepted for Publication: May 12, 2012.
Published Online: July 16, 2012. doi:10.1001/archinternmed.2012.3158
Author Contributions: Ms Chatterjee had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Chatterjee, Joynt, and Jha. Acquisition of data: Chatterjee and Jha. Analysis and interpretation of data: Chatterjee, Joynt, Orav, and Jha. Drafting of the manuscript: Chatterjee and Joynt. Critical revision of the manuscript for important intellectual content: Chatterjee, Joynt, Orav, and Jha. Statistical analysis: Chatterjee, Orav, and Jha. Study supervision: Jha.
Financial Disclosure: None reported.
2.Svensson S, Kjellgren KI, Ahlner J, Säljö R. Reasons for adherence with antihypertensive medication.
Int J Cardiol. 2000;76(2-3):157-16311104870
PubMedGoogle Scholar 3.Beach MC, Keruly J, Moore RD. Is the quality of the patient-provider relationship associated with better adherence and health outcomes for patients with HIV?
J Gen Intern Med. 2006;21(6):661-66516808754
PubMedGoogle Scholar 4.Sokol MC, McGuigan KA, Verbrugge RR, Epstein RS. Impact of medication adherence on hospitalization risk and healthcare cost.
Med Care. 2005;43(6):521-53015908846
PubMedGoogle Scholar 5.Ayanian JZ, Weissman JS, Schneider EC, Ginsburg JA, Zaslavsky AM. Unmet health needs of uninsured adults in the United States.
JAMA. 2000;284(16):2061-206911042754
PubMedGoogle Scholar 6.Boulware LE, Cooper LA, Ratner LE, LaVeist TA, Powe NR. Race and trust in the health care system.
Public Health Rep. 2003;118(4):358-36512815085
PubMedGoogle Scholar 7.Halbert CH, Armstrong K, Gandy OH Jr, Shaker L. Racial differences in trust in health care providers.
Arch Intern Med. 2006;166(8):896-90116636216
PubMedGoogle Scholar 8. Patient Protection and Affordable Care Act of 2010. Pub L No. 111-148. 124 Stat 119 through 124 Stat 1025
9.Centers for Medicare & Medicaid Services (CMS). HHS. Medicare program; hospital inpatient value-based purchasing program: final rule.
Fed Regist. 2011;76(88):26490-26547
Google Scholar 11.Bachrach D, Braslow L, Karl A. Toward a High Performance Health Care System for Vulnerable Populations: Funding for Safety-Net Hospitals. New York, NY: Commonwealth Fund; March 2012
14.American Hospital Association. FY2010 American Hospital Association Annual Survey. Chicago, IL: Health Forum; 2010
15.Jha AK, Orav EJ, Zheng J, Epstein AM. Patients' perception of hospital care in the United States.
N Engl J Med. 2008;359(18):1921-193118971493
PubMedGoogle Scholar 16.Jha AK, Li Z, Orav EJ, Epstein AM. Care in US hospitals—the Hospital Quality Alliance program.
N Engl J Med. 2005;353(3):265-27416034012
PubMedGoogle Scholar 17.Jha AK, Orav EJ, Li Z, Epstein AM. Concentration and quality of hospitals that care for elderly black patients.
Arch Intern Med. 2007;167(11):1177-118217563027
PubMedGoogle Scholar 18.Jha AK, DesRoches CM, Shields AE,
et al. Evidence of an emerging digital divide among hospitals that care for the poor.
Health Aff (Millwood). 2009;28(6):w1160-w117019858142
PubMedGoogle Scholar 19.Young GJ, Meterko M, Desai KR. Patient satisfaction with hospital care: effects of demographic and institutional characteristics.
Med Care. 2000;38(3):325-33410718357
PubMedGoogle Scholar 20.Lindenauer PK, Remus D, Roman S,
et al. Public reporting and pay for performance in hospital quality improvement.
N Engl J Med. 2007;356(5):486-49617259444
PubMedGoogle Scholar 21.Gaskin DJ, Hadley J. Population characteristics of markets of safety-net and non–safety-net hospitals.
J Urban Health. 1999;76(3):351-37012607901
PubMedGoogle Scholar 22.Kulik A, Shrank WH, Levin R, Choudhry NK. Adherence to statin therapy in elderly patients after hospitalization for coronary revascularization.
Am J Cardiol. 2011;107(10):1409-141421414595
PubMedGoogle Scholar 23.Roth MT, Esserman DA, Ivey JL, Weinberger M. Racial disparities in quality of medication use in older adults: findings from a longitudinal study.
Am J Geriatr Pharmacother. 2011;9(4):250-25821664193
PubMedGoogle Scholar 24.Frankenfield DL, Howell BL, Wei II, Anderson KK. Cost-related nonadherence to prescribed medication therapy among Medicare Part D beneficiaries with end-stage renal disease.
Am J Health Syst Pharm. 2011;68(14):1339-134821719594
PubMedGoogle Scholar 25.Weissman JS, Stern R, Fielding SL, Epstein AM. Delayed access to health care: risk factors, reasons, and consequences.
Ann Intern Med. 1991;114(4):325-3311899012
PubMedGoogle Scholar 26.Kaplan SH, Greenfield S, Ware JE Jr. Assessing the effects of physician-patient interactions on the outcomes of chronic disease.
Med Care. 1989;27(3):(suppl)
S110-S1272646486
PubMedGoogle Scholar 27.Alazri MH, Neal RD. The association between satisfaction with services provided in primary care and outcomes in type 2 diabetes mellitus.
Diabet Med. 2003;20(6):486-49012786685
PubMedGoogle Scholar 28.Andrews RM. United States. Agency for Healthcare Research and Quality: Serving the Uninsured: Safety-Net Hospitals, 2003. Rockville, MD: Agency for Healthcare Research and Quality; 2007
29.Goldman LE, Henderson S, Dohan DP, Talavera JA, Dudley RA. Public reporting and pay-for-performance: safety-net hospital executives' concerns and policy suggestions.
Inquiry. 2007;44(2):137-14517850040
PubMedGoogle Scholar 30.Jha AK, Orav EJ, Epstein AM. The effect of financial incentives on hospitals that serve poor patients.
Ann Intern Med. 2010;153(5):299-30620820039
PubMedGoogle Scholar 31.Werner RM, Goldman LE, Dudley RA. Comparison of change in quality of care between safety-net and non–safety-net hospitals.
JAMA. 2008;299(18):2180-218718477785
PubMedGoogle Scholar 32.Ross JS, Cha SS, Epstein AJ,
et al. Quality of care for acute myocardial infarction at urban safety-net hospitals.
Health Aff (Millwood). 2007;26(1):238-24817211034
PubMedGoogle Scholar 33.Popescu I, Werner RM, Vaughan-Sarrazin MS, Cram P. Characteristics and outcomes of America's lowest-performing hospitals: an analysis of acute myocardial infarction hospital care in the United States.
Circ Cardiovasc Qual Outcomes. 2009;2(3):221-22720031841
PubMedGoogle Scholar 34.Goldman LE, Vittinghoff E, Dudley RA. Quality of care in hospitals with a high percent of Medicaid patients.
Med Care. 2007;45(6):579-58317515786
PubMedGoogle Scholar 35. McHugh M, Kang R, Hasnain-Wynia R. Understanding the safety net: inpatient quality of care varies based on how one defines safety-net hospitals.
Med Care Res Rev. 2009;66(5):590-60519398722
PubMedGoogle Scholar