Evaluation of Changes in Veterans Affairs Medical Centers’ Mortality Rates After Risk Adjustment for Socioeconomic Status

Key Points Question Does risk adjustment for socioeconomic status change Veterans Affairs Medical Centers’ rankings for mortality rates after hospitalization for heart failure and pneumonia? Findings In this cross-sectional analysis, Veterans Affairs medical centers’ mortality rates were highly correlated in models that included or did not include socioeconomic factors, and few medical centers’ rates were associated with substantial changes in risk-adjusted mortality after adjustment for socioeconomic status. Meaning This study’s finding suggest that Veterans Affairs medical centers’ performance on mortality measures after hospitalization for heart failure and pneumonia were minimally changed after risk adjustment for socioeconomic factors.


Introduction
Nearly all hospitals in the United States, including all Veterans Affairs medical centers (VAMCs), report mortality rates for patients hospitalized for common medical and surgical conditions. 1 Performance on these outcome measures carries high stakes. The Centers for Medicare & Medicaid Services (CMS) adjust payments to hospitals based on their performance on quality-of-care measures, including hospital mortality rates after acute myocardial infarction, heart failure (HF), and pneumonia. 2 Hospital mortality rates inform the Veterans Affairs (VA) Strategic Analytic and Information Learning ratings, which the VA health system uses to measure performance in VAMCs. 3 The Veterans Access, Choice, and Accountability Act of 2014 requires the VA health system to release comprehensive data to the public on access to, quality of, and outcomes of care, including hospital mortality.
Valid hospital outcome measures must adequately account for differences in patients' risk. 1 Much of the prior concern with risk adjustment has involved the source of the data (administrative vs clinical records), the selection of appropriate covariates, or the optimal approach to statistical modeling. [4][5][6] Fewer studies have examined the role of socioeconomic status (SES) and other social risk factors in risk adjustment, although these factors are associated with worse postdischarge outcomes and vary markedly across private sector hospitals. [7][8][9] Furthermore, prior studies have documented concerns about the validity of claims-based measures of clinical risk. 10,11 Socioeconomic factors may capture heterogeneity in clinical risk that is not correlated with claims-based measures of severity. Despite emerging consensus from stakeholder groups that profiling performance should include consideration of patients' sociodemographic characteristics, neither CMS nor the VA health system include sociodemographic factors other than age and sex in risk-adjusted models of hospital mortality. 9, [12][13][14][15] The 21st Century Cures Act requires CMS to adjust financial penalties (but not riskstandardized readmission rates) based on the hospitals' proportion of patients with dual Medicaid and Medicare eligibility. 16 Prior studies have found conflicting evidence on the association of socioeconomic risk adjustment with hospital rankings. [16][17][18][19][20][21] These studies have focused exclusively on readmissions and most have used claims-based conditions, rather than objective clinical data, to account for severity of illness. The association of adjustment for socioeconomic factors with assessments of hospital mortality is not well understood.
The objective of this study was to develop and test risk-adjustment approaches that incorporate veterans' sociodemographic characteristics (neighborhood disadvantage, income, Medicaid enrollment, homelessness, race/ethnicity, and nursing home residence) into assessments of hospital mortality. We conducted a retrospective, observational, cross-sectional study that tested the performance of models that did or did not include sociodemographic characteristics and assessed the association of the inclusion of sociodemographic characteristics and objective clinical covariates with the rankings of VAMCs for hospital mortality after HF and pneumonia. to report model discrimination and hospital-level risk-adjusted mortality. We randomly sampled 1 hospitalization for veterans with more than 1 admission from 2012 to 2014. [22][23][24] This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline for cross-sectional studies. The Providence VAMC Institutional Review Board approved the study protocol and waived the need for informed consent because the risk to the study participants was no more than minimal and the research could not practicably be conducted without the waiver.

Data Sources and Study Population
Consistent with VA health system and CMS specifications, we excluded hospitalizations from veterans with less than 12 months of continuous VA health system enrollment in the year prior to the admission, those discharged alive within 24 hours, those who left against medical advice, and those with hospice use within 1 year prior to the date of admission (eFigure 1 in the Supplement). For patients who were transferred between VAMCs or from a VAMC to a private sector hospital, the outcome was associated with the hospital in which the veteran was initially admitted. [22][23][24]

Measures
The primary outcome variable was death from any cause within 30 days after hospital admission, as derived from the VA health system vital status file. 25 Deaths could occur in the hospital or after discharge.
Covariates from the CMS and VA health system model included age, sex, and claims-based comorbid conditions (22 for HF and 30 for pneumonia) derived from the index admission, as well as outpatient and inpatient encounter data for the 12 months prior to admission (eTable 2 in the Supplement). We did not include comorbid conditions from Medicare claims, consistent with the VA health system's methods. 26 Objective clinical variables were selected based on prior literature, clinical face validity, availability at the time of presentation, and the feasibility of routine data extraction. 10 These variables included systolic blood pressure, diastolic blood pressure, heart rate, respiratory rate, oxygen saturation, body mass index, hematocrit, serum sodium level, serum potassium level, blood urea nitrogen level, serum creatinine level, serum glucose level (pneumonia only), B-type natriuretic peptide level (HF only), and ejection fraction (HF only). Each variable was derived during the 2 days prior to admission until 1 day after admission using the value closest to the admission time. 10 We used natural language processing methods to extract ejection fraction for patients with HF. 27 Rates of missing values are shown in eTable 3 in the Supplement.
We included 9 sociodemographic covariates: the Area Deprivation Index (ADI), race/ethnicity, veteran's priority group, Medicaid enrollment, original reason for Medicare eligibility, homelessness, prior nursing home use in the year before admission, direct admission from a nursing home, and rural residence. Medicaid enrollment and original reason for Medicare enrollment were not available for persons without Medicare coverage, and were therefore limited to analyses of veterans aged 66 years or older.
The ADI is a factor-based composite score of 17 poverty, education, housing, and employment variables derived from the US Census that can be aggregated to the census block or census tract. 28,29 The measure is constructed with a mean of 100 and standard deviation of 10 nationally. Because it is a relative measure of disadvantage, we assigned each veteran to deciles of the ADI. We used veterans' census tracts (available in VA Planning Systems Support Group data) to derive their ADI score. For veterans with missing census tracts, we used 9-digit zip codes to link to census block group. If the 9-digit zip code was missing, we used veterans' 5-digit zip codes. In the validation cohort, the source of the ADI was as follows: 81% from the census tract, 15% from the 9-digit zip code, and 3% from the 5-digit zip code; 1% were missing (eTable 3 in the Supplement). For homeless

JAMA Network Open | Health Policy
Changes in VA Hospital Mortality Rates After Adjustment for Socioeconomic Status veterans and nursing home residents, we derived the ADI using the last recorded address in VA health system enrollment records. The VA health system assigns enrollment priority group on the basis of the presence and degree of service-related disability, income, features of military service, eligibility for Medicaid coverage, and receipt of VA pension benefits, among other factors. For nursing home residence, we constructed a 3-level variable of short-term or long-term nursing home residence on the basis of whether the veteran had no evidence of nursing home use, was in a nursing home for less than 90 days, or was in a nursing home for greater than 90 days in the 1 year prior to the index admission. We also included a binary indicator of whether the hospital admission occurred directly from a nursing home. Although enrollment priority and nursing home residence can reflect health status, we classified them as socioeconomic factors because enrollment priority includes measures of income, assets, and Medicaid eligibility, and nursing home residence reflects patients' living conditions. Rurality was captured using a rural-urban classification system developed by the VA Planning Systems Support Group in collaboration with the US Census Bureau. Homelessness was defined by the presence of an ICD-9-CM code of V60.0 or V.60.1. 30 Last, we included race/ethnicity (non-Hispanic White, non-Hispanic Black, Hispanic, and other). 31

Statistical Analysis
Statistical analysis was performed from March 1, 2019, to April 1, 2020. We constructed hierarchical logistic regression models to estimate 30-day mortality after including the CMS and VA health system claims-based, clinical, and socioeconomic covariates. The models include adjustments for facilityspecific intercepts using the hierarchical structure to account for clustering of patients and differential sample sizes within hospitals, consistent with the approach used by the CMS and VA health system. We imputed missing continuous covariates using Markov chain Monte Carlo single imputation. For observations with missing ADI, we imputed the continuous value before assigning the observation to a decile. Observations with missing race/ethnicity (n = 222 [0.5%] for HF; n = 250 [0.6%] for pneumonia) were dropped. The discrimination of each model was reported using the area under the receiver operating characteristic curve (C statistic).
Using the Spearman correlation, we tested the agreement in hospital rankings from models that included or did not include socioeconomic covariates. We estimated the number of hospitals that changed their ranking by more than 10% (ie, 13 ranking positions) and that moved across a quintile threshold after the inclusion of socioeconomic covariates. We report stratified analyses of the association of socioeconomic adjustment for VA hospitals in the highest, lowest, and middle 3 quintiles of risk-standardized mortality (as derived from the CMS and VA health system claims-based model). Last, we repeated all analyses after restricting to patients 66 years or older with Medicare coverage. All analyses were conducted in SAS, version 9.4 (SAS Institute Inc).

Characteristics of the Study Population
Among 42 892 veterans hospitalized with HF in 131 medical centers from 2012 to 2014, the mean (SD) age was 71.9 (11.4) years, 98.2% were male, 23.3% were Black, the mean (SD) number of comorbid conditions was 5.9 (3.0), 13.5% received nursing home care prior to admission, 11.2% had evidence of homelessness, and the mean (SD) ADI was 105.2 (13.8) ( Table 1). Among 39 062 veterans hospitalized with pneumonia in 131 medical centers from 2012 to 2014, the mean (SD) age was 71.0 (12.4) years, 96.8% were male, 15.6% were Black, the mean (SD) number of comorbid conditions was 5.1 (3.1), 13.7% received nursing home care prior to admission, 12.6% had evidence of homelessness, and the mean (SD) ADI was 105.4 (13.4). The unadjusted 30-day mortality rate was 7.2% for HF and 8.1% for pneumonia; 64.7% (HF) and 56.6% (pneumonia) of deaths occurred after discharge. The prevalence of claims-based comorbidities is shown in eTable 2 in the Supplement. At the hospital level, the interquartile ranges (IQRs) of the proportion of Black patients were 4.3% to 26.9% for HF and 3.6% to 20.3% for pneumonia (eTable 4 in the Supplement). The IQRs for the proportion of rural patients were 19.0% to 56.6% (HF) and 17.0% to 55.7% (pneumonia), and for the proportion of homeless individuals, they were 6.0% to 13.3% (HF) and 7.5% to 16.1% (pneumonia). The IQRs for hospital-level mean ADI score were 102.5 to 110.5 (HF) and 102.2 to 110.1 (pneumonia).
eTable 5 in the Supplement shows unadjusted 30-day mortality rates by socioeconomic characteristics.

Change in Mortality Rates and Rankings After Adjustment for SES
For both HF and pneumonia, the Spearman correlation in ranking between the CMS claims-based approach and an approach using the CMS claims and social risk factors was 0.98 (Figure 1 and With the use of the CMS model for HF, the mean (SD) risk-standardized mortality rate of VAMCs during the period from 2012 to 2014 was 6.0% (0.4%) in the lowest quintile, 7.2% (0.4%) in the middle 3 quintiles, and 8.8% (0.6%) in the highest quintile ( Table 3). After adding socioeconomic covariates, the mean (SD) risk-standardized mortality rate for these hospitals in 2012-2014 was 6.1%    absolute change in risk-adjusted mortality was 0.08 percentage points (range, 0.02-0.13 percentage points) ( Table 3).
Using CMS's model for pneumonia, the mean (SD) risk-standardized mortality rates for VAMCs was 6.6% (0.4%) in the lowest quintile, 8.1% (0.6%) for those in the middle 3 quintiles, and 10.0% (0.8%) for those in the highest quintile (Table 3). After adding socioeconomic covariates, the mean risk-standardized mortality rates for these hospitals were unchanged. Among VAMCs in the highest quintile for pneumonia mortality, 1 of 26 (3.8%) changed by more than 13 ranking positions. Sixteen VAMCs changed by 1 quintile after SES adjustment, with 13 of these hospitals in the middle 3 quintiles of mortality rates. Similar findings were observed for analyses that included clinical covariates  (eTable 6 in the Supplement) and were limited to veterans aged 66 or older who were enrolled in Medicare (eTable 7 in the Supplement).

Discussion
This study of changes in risk-adjusted mortality in VAMCs after adjusting for socioeconomic characteristics has 4 main findings. First, risk-adjustment models that included socioeconomic factors only minimally improved the discrimination of 30-day mortality when added to models that included CMS claims-based or novel clinical covariates. Second, the rankings of VAMCs with or without adjustment for socioeconomic characteristics were tightly correlated, with all correlation coefficients at or above 0.98. Third, few VAMCs with high mortality rates would be reclassified if socioeconomic characteristics were included in the current VA health system and CMS risk- VAMCs that changed their quintile ranking were largely in the middle of the distribution of riskadjusted mortality and had minimal absolute changes in adjusted mortality rates.
Our findings align with those of prior work showing that adjustment for social risk factors produced little change in hospitals' rankings in readmission rates. [17][18][19]32 Ross et al 33 also reported that, among patients with Medicare fee-for-service coverage, safety-net hospitals had similar riskadjusted mortality rates compared with non-safety-net hospitals. We extend this work in 3 main ways. First, we consider a broader range of social risk factors, including assessing neighborhood disadvantage at a more granular census block level rather than the 4-digit zip code or county level, using a composite of 17 measures of social risk. 34 The use of a composite measure may capture social risk in a more comprehensive manner than the use of area-level income or poverty alone. Second, in contrast to studies of Medicare data that adjust for clinical risk using claims-based conditions, we evaluated the association of social risk factors in models that also included laboratory test values, vital signs, and ejection fraction, important factors associated with hospital mortality. Our study suggests that after including these novel clinical covariates in risk-adjustment models, adding social risk factor variables did not meaningfully change VAMCs' risk-standardized mortality rates. Third, while other studies have focused on readmission, our analysis examined mortality, a widely reported hospital quality measure with evidence of disparities by SES. [35][36][37] Prior work has found that hospitals' risk-standardized mortality rates exhibit weak or no association with risk-standardized readmission rates. 38 Our findings contrast with those of studies that report substantial changes in readmissions penalties and reduced variations in outcomes after adjusting for social risk factors. 16 39,45,46 This finding suggests that adjustments for social risk factors may have less association with changing assessments of VAMC outcomes compared with other health systems and payers. Third, it is possible that adjustment for social risk factors has less of an association with hospital mortality than with readmission.
The proliferation of public reporting and value-based payment initiatives has focused attention on the impact of these programs on hospitals that disproportionately serve patients with greater social disadvantage and the need to consider socioeconomic factors in measuring risk-adjusted performance rates. [12][13][14][15] Failure to consider these factors may penalize hospitals based on the patients they serve rather than the quality of care they deliver, and therefore place further financial stress on safety-net hospitals and potentially widen disparities in care. [12][13][14] However, some critics have expressed concern that adjustments for social risk factors may inappropriately set a lower standard of care in hospitals that serve socially disadvantaged patients and mask disparities by adjusting them away. [12][13][14] Our study has 4 policy implications in the context of this ongoing debate. First, our study indicates that, at least in the case of hospital mortality measures for HF and pneumonia, adjusting for socioeconomic factors did not yield substantial changes in adjusted mortality rates for VAMCs.