Quality of Care and Patient Outcomes in Critical Access Rural Hospitals

Context Critical access hospitals (CAHs) play a crucial role in the US rural safety net. Current policy efforts have focused primarily on helping these small, isolated hospitals remain financially viable to ensure access for individuals living in rural areas in the United States; however, little is known about the quality of care they provide or the outcomes their patients achieve.

Objectives To examine the quality of care and patient outcomes at CAHs and to understand why patterns of care might differ for CAHs vs non-CAHs.

Design, Setting, and Patients A retrospective analysis in 4738 US hospitals of Medicare fee-for-service beneficiaries with acute myocardial infarction (AMI) (10 703 for CAHs vs 469 695 for non-CAHs), congestive heart failure (CHF) (52 927 for CAHs vs 958 790 for non-CAHs), and pneumonia (86 359 for CAHs vs 773 227 for non-CAHs) who were discharged in 2008-2009.

Main Outcome Measures Clinical capabilities, performance on processes of care, and 30-day mortality rates, adjusted for age, sex, race, and medical comorbidities.

Results Compared with other hospitals (n = 3470), 1268 CAHs (26.8%) were less likely to have intensive care units (380 [30.0%] vs 2581 [74.4%], P < .001), cardiac catheterization capabilities (6 [0.5%] vs 1654 [47.7%], P < .001), and at least basic electronic health records (80 [6.5%] vs 445 [13.9%], P < .001). The CAHs had lower performance on processes of care than non-CAHs for all 3 conditions examined (concordance with Hospital Quality Alliance process measures for AMI, 91.0% [95% CI, 89.7%-92.3%] vs 97.8% [95% CI, 97.7%-97.9%]; for CHF, 80.6% [95% CI, 79.2%-82.0%] vs 93.5% [95% CI, 93.3%-93.7%]; and for pneumonia, 89.3% [95% CI, 88.6%-90.0%] vs 93.7% [95% CI, 93.6%-93.9%]; P < .001 for each). Patients admitted to CAHs had higher 30-day mortality rates for each condition than those admitted to non-CAHs (for AMI: 23.5% vs 16.2%; adjusted odds ratio [OR], 1.70; 95% confidence interval [CI], 1.61-1.80; P < .001; for CHF: 13.4% vs 10.9%; adjusted OR, 1.28; 95% CI, 1.23-1.32; P < .001; and for pneumonia: 14.1% vs 12.1%; adjusted OR, 1.20; 95% CI, 1.16-1.24; P < .001).

Conclusion Compared with non-CAHs, CAHs had fewer clinical capabilities, worse measured processes of care, and higher mortality rates for patients with AMI, CHF, or pneumonia.

Critical access hospitals (CAHs) play an important and unique role in the US health care system, caring for individuals who live in rural areas and who might otherwise have no accessible inpatient care. This hospital designation, created by the Medicare Rural Hospital Flexibility Program of the 1997 Balanced Budget Act, resulted from a federal effort to increase resources for small, geographically isolated hospitals, many of which were struggling financially. The bill defined CAHs as hospitals with no more than 25 acute care beds and located more than 35 miles from the nearest hospital.¹ Hospitals that converted to CAH status became eligible for cost-based reimbursement rather than diagnosis related group–based reimbursement.¹ As a result, margins improved and closures among these small rural hospitals decreased dramatically^2,3; more than a quarter of the acute care hospitals in the United States now have the CAH designation.

The CAH designation was created with the goal of ensuring “proximate access” to basic inpatient and emergency care close to home for approximately 20% of the US population that still lives in rural communities.⁴ The program has been highly successful in protecting access to inpatient care for rural communities, while providing care that receives high scores on patient satisfaction.⁵ However, despite broad policy interest in helping CAHs provide access to inpatient care, little is known about the quality of care they provide—these hospitals are exempt from reporting to both the Joint Commission performance measure program⁶ and the Hospital Quality Alliance (HQA) national public reporting program.⁷ We are unaware of recent national data comparing outcomes at these hospitals to a national sample. Critical access hospitals have less access to capital and fewer health care professionals in their communities, including fewer specialists.⁸ Therefore, these hospitals may face equal or greater challenges in delivering high-quality care⁹ compared with other vulnerable hospitals, such as safety net hospitals, which have been more extensively studied.¹⁰ Understanding whether the CAH designation has been helpful, in not only improving access, but also in ensuring high-quality care, is a key element in evaluating federal efforts to ensure an effective rural health system.

We sought to examine CAHs' clinical and personnel resources, the quality of care they deliver, and their patients' outcomes. We focused on 3 common conditions—congestive heart failure (CHF), acute myocardial infarction (AMI), and pneumonia. We also sought to identify what factors, such as clinical capabilities, size, patient volume, or other related issues, might explain any differences in outcomes of care.

We used the Medicare Provider Analysis and Review file to identify nonfederal hospitals that provide acute care services to Medicare beneficiaries in the 50 US states or District of Columbia. We used the 2009 American Hospital Association survey to obtain data on hospital characteristics, including critical access designation, size, ownership, teaching status, and region. We linked these data with the 2009 Area Resource File, which contains county-level data on median household income and poverty rate. Although the original legislation specified that only isolated rural hospitals qualified for CAH status, states subsequently granted exemptions for this rule, allowing some hospitals in suburban or even urban settings to be eligible. Therefore, we linked the rural urban commuting area codes, which detail population density and urbanization at a granular level, to examine the degree to which rurality affected our findings.¹¹

We defined our study population as Medicare fee-for-service beneficiaries admitted to the hospitals in our sample in 2008-2009 with a primary discharge diagnosis of AMI, CHF, or pneumonia (International Classification of Diseases, Ninth Revision, Clinical Modification [ICD-9-CM ] codes 398.91, 404.x1, 404.x3, 428.0-428.9 for CHF; ICD-9-CM codes 410.xx, excluding 410.x2, for AMI; and ICD-9-CM codes 480-486 for pneumonia). Patient race as reported to Medicare was categorized based on self-report. We followed the Centers for Medicare & Medicaid Services approach for classifying “index admissions,”^12,13 allowing patients to be included in the sample more than once. All patients were assigned to the admitting hospital regardless of whether they were transferred. Our final patient population included 2 351 701 admissions across these 3 conditions.

We used the American Hospital Association survey to quantify resources that have been associated with better care,^14,15 including the presence of an intensive care unit, the ability to perform cardiac catheterization or surgery, and nurse staffing levels. Nurse staffing was estimated by calculating the number of full-time equivalent nurses on staff per 1000 patient-days.^16,17 We used the Area Resource File to estimate the total physician and subspecialist supply for the county in which each hospital was located. Each hospital's health information technology resources were determined from the American Hospital Association Health Information Technology survey, which was distributed to every acute care hospital in the United States in 2009. The survey asked responding hospitals to report the degree of adoption of specific electronic health record (EHR) functions and achieved a response rate of 63%.¹⁸

We used HQA data to obtain hospitals' performance on process measures for AMI, CHF, and pneumonia during 2009. Because of sample size cutoffs and reporting exemptions, these measures were available for only a subset of the hospitals in our sample. We calculated an overall performance score for each hospital for each condition¹⁹ (eTable 1). We used Medicare data to calculate mortality within 30 days of admission. Each patient's likelihood of death was adjusted for age, sex, race, and medical comorbidities using the Centers for Medicare & Medicaid Services Hierarchical Condition Category mortality models,²⁰ which were developed by the Centers for Medicare & Medicaid Services and have been demonstrated in recent studies to have a better C statistic and predictive accuracy than the Charlson and Elixhauser methods.²¹

We compared summary statistics for hospital characteristics, demographics, and patient comorbidities between CAHs and non-CAHs using χ² tests and t tests or Wilcoxon rank sum tests as appropriate. We used χ² tests to compare the presence of each clinical resource and functionality between CAHs and non-CAHs. We analyzed performance on the HQA metrics, weighting each hospital's performance by its number of patients with that diagnosis. We then created patient-level logistic regression models for 30-day mortality, clustered at the hospital level.

We subsequently built multivariable regression models. We first adjusted for factors that are outside the control of the hospitals and policy makers, including region, hospital ownership, and median county income. We next added variables to the model that we postulated might be in the explanatory pathway between CAH status and outcomes, and might be amenable to change by either hospitals or policy makers, in a stepwise fashion, first adding measures of clinical personnel, followed by clinical resources and system membership, the presence of an EHR, and annual condition-specific case volume. In addition, we examined models adjusting for rurality using the rural urban commuting area codes (divided into urban [≥50 000 population], large town [10 000-49 999 population], small town [2500-9999 population], and rural [<2500 population] categories). Although rurality is highly collinear with being a CAH, it may also be associated with other, unmeasured (or inadequately measured) factors, including travel time and quality of clinical personnel and resources.

Because CAHs transfer more patients than non-CAHs, we examined in sensitivity analyses differences in mortality rates after excluding all transfers. To better understand whether differences in outcomes between CAHs and other hospitals were driven primarily by size and rural status, or whether these differences might be driven by other factors such as CAHs' exemption from reporting or payment mechanisms, we conducted additional sensitivity analyses and restricted our sample to small, rural hospitals. In addition, we used established methods^3,4 to model the degree of association between an unmeasured confounder and both our primary predictor (CAH status) and our outcome (mortality) that would have had to be present to eliminate our findings.

To account for multiple comparisons, we considered 2-sided P < .008 to be statistically significant. Analyses were performed by using SAS version 9.2 (SAS Institute Inc, Cary, North Carolina).

Of the 4738 hospitals providing acute care to Medicare beneficiaries in 2008-2009, 1268 (26.8%) were designated as CAHs. The median number of operational beds in a CAH was 18 (interquartile range [IQR], 14-23) beds compared with 82 (IQR, 41-154) beds for non-CAHs (Table 1). Critical access hospitals were more likely to be publicly owned and less likely to be a teaching hospital, located in an urban area or large town, or part of a hospital system. Nearly half of the CAHs (49.0%) were located in the Midwest. In general, CAHs were located in counties with a lower median income than non-CAHs and served a higher proportion of Medicare patients but a lower proportion of Medicaid patients.

We identified 2 351 701 index admissions for patients with CHF, AMI, or pneumonia during our study period, of which 149 989 admissions were to CAHs. Patients admitted to CAHs tended to be older and had a higher incidence of diabetes and depression, but a lower incidence of hypertension and chronic kidney disease (Table 2). Patients admitted to CAHs were more likely to be transferred to another acute care hospital than those admitted to non-CAHs (29.7% vs 9.5% for AMI, 7.4% vs 2.5% for CHF, and 5.6% vs 1.5% for pneumonia; P < .001 for each) and had significantly shorter lengths of stay for all 3 conditions. Patients admitted to CAHs were less likely to be transferred to a hospice at the time of discharge than patients admitted to non-CAHs.

Critical access hospitals had fewer clinical resources than other hospitals and were less likely to have intensive care units, cardiac catheterization capability, or the ability to perform surgeries (Table 3). Critical access hospitals had comparable nurse staffing levels to non-CAHs, but were located in counties with fewer specialists, with a 7-fold difference in the supply of cardiologists and pulmonologists per 100 000 population.

Critical access hospitals were less likely to have the key functions that comprise an EHR. Overall, 6.5% of CAHs had at least a basic EHR compared with 13.9% of non-CAHs.¹⁸ Each EHR component, including clinical documentation, test results viewing, computerized physician order entry, and decision support, was present less often at CAHs, and we found no difference in adoption of telemedicine (Table 4). Critical access hospitals were also less likely to be exchanging clinical data electronically with other hospitals or with outpatient practices.

For all 3 conditions, CAHs had lower performance on HQA measures than non-CAHs did among reporting hospitals (Figure). For patients admitted with AMI, CAHs provided care that was concordant with HQA process measures 91.0% (95% confidence interval [CI], 89.7%-92.3%) of the time compared with 97.8% (95% CI, 97.7%-97.9%) of the time for non-CAHs (difference of 6.8%, P < .001). The difference was larger for CHF (12.9%; 80.6% [95% CI, 79.2%-82.0%] vs 93.5% [95% CI, 93.3%-93.7%]; P < .001) and smaller but still significant for pneumonia (4.4%; 89.3% [95% CI, 88.6%-90.0%] vs 93.7% [95% CI, 93.6%-93.9%]; P < .001). For 14 of the 17 individual measures, CAHs performed worse than non-CAHs (eTable 2). These differences persisted after adjusting for case mix and hospital characteristics (eTable 3).

Patients admitted to CAHs had higher 30-day risk-adjusted mortality rates for all 3 conditions than patients admitted to non-CAHs (Table 5). Patients admitted to CAHs had 7.3% higher absolute 30-day mortality rates for AMI (23.5% vs 16.2%; adjusted odds ratio [OR], 1.70; 95% CI, 1.61-1.80; P < .001); 2.5% higher mortality rates for CHF (13.4% vs 10.9%; adjusted OR, 1.28; 95% CI, 1.23-1.32; P < .001); and 2% higher mortality rates for pneumonia (14.1% vs 12.1%; adjusted OR, 1.20; 95% CI, 1.16-1.24; P < .001) than those admitted elsewhere. When we limited our analyses to patients who were not transferred, our results were similar (eTable 4).

We next built models that serially adjusted for variables that might be in the explanatory pathway to identify potentially actionable differences between CAHs and non-CAHs that contribute to outcomes. We found that differences in clinical personnel and resources into the model slightly attenuated the findings (Table 5); even after fully adjusting for all variables, including rurality, differences in mortality between CAHs and non-CAHs persisted for AMI (adjusted OR, 1.18; 95% CI, 1.09-1.28; P < .001) and for CHF (adjusted OR, 1.15; 95% CI, 1.00-1.31; P = .04), but not pneumonia (adjusted OR, 0.92; 95% CI, 0.87-0.97; P = .003).

When we limited our analyses to small, rural hospitals (1022 [81%] of the CAHs and 379 [11%] of the non-CAHs), we found differences in resources, quality of care, and outcomes between those with vs without the CAH designation (eTable 5 and eTable 6). Although there were no significant differences in measured quality for AMI (mean HQA summary score, 89.7 vs 90.3; P = .59), CAHs had higher mortality than non-CAHs for this condition (adjusted OR, 1.14; 95% CI, 1.05-1.24; P = .003). Critical access hospitals had lower performance on quality measures and higher mortality for CHF (mean HQA summary score, 78.7 vs 84.8; P < .001; adjusted OR, 1.09; 95% CI, 1.03-1.16; P = .003), and lower performance on quality measures but identical mortality for pneumonia (mean HQA summary score, 88.7 vs 91.1; P < .001; adjusted OR, 1.05; 95% CI, 0.99-1.11; P = .11) (Table 6).

Based on our sensitivity analysis, unmeasured confounding was unlikely to explain our findings. For AMI, for example, if an unmeasured confounder tripled mortality risk (a much stronger predictor of mortality than any of our current comorbidities) and was 3 times more common in patients in CAHs than in non-CAHs, the OR for mortality associated with receiving care at a CAH would decrease to 1.44, still statistically significant and clinically meaningful.

Despite more than a decade of concerted policy efforts to improve rural health care, our findings suggest that substantial challenges remain. Although CAHs provide much-needed access to care for many of the nation's rural citizens, we found that these hospitals, with their fewer clinical and technological resources, less often provided care consistent with standard quality metrics and generally had worse outcomes than non-CAHs. The absolute differences in outcomes were even larger than those outcomes reported in the initial work on this topic by Keeler et al,²² who demonstrated an excess all-cause mortality of 1.4% in rural hospitals using data from the 1980s, and comparable with differences noted by the Medicare Payment Advisory Committee using data from 2003.² These findings suggest that efforts to date have been insufficient in improving the quality of inpatient care in rural communities²³—and indicate a need for greater policy attention to the challenges these hospitals face. Given that CAHs care for a population that tends to be older and less likely to have routine access to primary care services,²³ it is particularly important that policy efforts help CAHs meet these challenges.

The CAH designation, created with the goal of preserving access to care for individuals living in rural areas in the United States, directed financial resources to vulnerable rural hospitals at a time when many were closing due to financial insolvency. A number of regulations intended to promote quality were included in the legislation, including a formal requirement for credentialing and a state-run evaluation of quality. In return, designation as a CAH provided hospitals with financial security through cost-based reimbursement, which led to significant improvement in these hospitals' financial stability and allowed them to remain open, preserving access^9,24,25 while maintaining patient satisfaction scores equal to or greater than those of non-CAHs.⁵ However, our findings suggest that these efforts have been insufficient in ensuring high-quality care.

Critical access hospitals had significantly poorer performance on process measures, which may be due to fewer resources to devote to quality improvement. Because CAHs are not required to report HQA data,⁷ the CAHs that reported (which ranged from 39% of CAHs for AMI to 71% of CAHs for pneumonia) may represent a higher-performing subset of CAHs than those choosing not to, which would understate the true differences in care. Furthermore, CAHs have typically been exempt from pay-for-performance programs in the past and will likely be excluded from national value-based purchasing efforts at least in the near term.^26-28 Engaging in the process of collecting and reporting data is an important step toward developing an internal quality improvement strategy.²⁹ Indeed, the Institute of Medicine has recommended that all CAHs participate in the HQA program for this reason.³⁰

We found that personnel and clinical resources explained some of the mortality differences between CAHs and other hospitals. Ensuring adequate personnel and resources is challenging for CAHs,^9,25 given their difficulties in recruiting health care practitioners.²⁵ Inadequate outpatient care including lack of access to ongoing primary care, posthospitalization follow-up, rehabilitation, and home-based care^31-33 may also contribute to poorer outcomes. Additional policy efforts to bring needed health care practitioners to underserved areas to ensure that CAHs have key clinical resources may be helpful. Given prior evidence that being a member of a hospital system may be related to improved clinical outcomes,^34,35 promoting partnerships with health care systems might be a useful strategy to help CAHs. Such partnerships could include onsite rotations by clinicians with specialty training, increased use of telemedicine, or formal referral and transfer agreements—arrangements that allow patients to remain close to home while still facilitating access to specialty care are likely to be particularly well received by patients. One approach might be to provide financial incentives for tertiary care hospitals to partner with CAHs, potentially tying incentives to the CAH's performance on quality metrics.

Although we did not find that the presence of an EHR explained a significant amount of the difference in clinical outcomes between CAHs and non-CAHs, this area warrants extra attention. The use of technology, particularly telemedicine and clinical data exchange, has important applications in underserved areas.^36-39 Critical access hospitals lack financial capital and access to the personnel needed to install and effectively maintain these systems.^40,41 The federal effort to promote EHR adoption among CAHs has focused on technical assistance by the Regional Extension Centers.⁴² However, some Regional Extension Centers have elected not to work with CAHs and others are charging fees that may be unaffordable for CAHs. Policy makers may need to consider additional strategies to avoid exacerbating an already emerging digital divide.¹⁸

Adding rurality to our models seemed to explain some of the mortality differences we observed, and when we compared small, rural CAHs to small, rural non-CAHs, the excess mortality at CAHs decreased. These findings may not be surprising because poorer outcomes in rural settings was part of the motivation for creation of the critical access designation. Our findings suggest that a substantial proportion of the barriers faced by CAHs are due to their size and their rural location, even after accounting for other factors such as clinical resources and personnel. Rurality is likely associated with other unmeasured factors, such as travel distances to primary care or hospital, that affect outcomes. Better understanding what factors are closely associated with rurality that may help explain some of the differences in outcomes would be helpful in formulating effective interventions to help CAHs.

Despite the significant policy attention directed toward these vulnerable hospitals, there has been little empirical work on quality of care in a national sample of CAHs. Lutfiyya et al⁴³ examined performance on HQA process measures in 2004, the first year for which these data were available, and found that CAHs had a lower performance than non-CAHs did. More recent comparisons have shown mixed results. Some studies^44,45 have found that rural hospitals provide lower quality care; however, another study⁴⁶ that examined self-selected hospitals engaged in national quality improvement programs failed to find a difference. Using 2003 data, compared with other rural hospitals, the Medicare Payment Advisory Committee found that CAHs had higher risk-adjusted mortality rates for CHF, AMI, pneumonia, stroke, and gastrointestinal hemorrhage. Our findings extend the Medicare Payment Advisory Committee work by focusing on a contemporary sample and a comparison group of nonrural hospitals, and by assessing care across a wide range of metrics while accounting for hospital characteristics and resources.²

Our study has limitations. We used administrative data, which fail to capture important clinical and patient characteristics (such as educational attainment) that likely affect outcomes. Based on our sensitivity analysis, however, it is unlikely that any unmeasured confounder could be strong enough to fully account for the difference between CAHs and non-CAHs outcomes. We lacked data on the experience or qualifications of the clinicians caring for patients at CAHs, which could have potentially explained some of our findings. We were also unable to assess the role of patient choice in patterns of care—patients may have declined transfer for more advanced care due to personal preference even if clinicians recommended that a transfer occur. We could not examine outpatient care and thus were unable to assess to what extent these differences might affect our findings. Because we relied on Medicare fee-for-service data for outcomes, we could not assess whether the patterns observed were also true for Medicare Advantage patients or for younger patients. In addition, mortality may be a crude measure of hospital quality; therefore, we attempted to incorporate both structural and process measures to provide a more comprehensive view of care at CAHs.

In summary, CAHs play an essential role in ensuring access to health care for individuals living in rural areas in the United States. However, these institutions face many challenges, remain underresourced in terms of both clinical and technological capabilities, perform worse on process measures, and have higher mortality rates than non-CAHs. More than a decade after major federal and state efforts to save US rural hospitals, these findings should be seen as a call to focus on helping these hospitals improve the quality of care they provide so that all individuals in the United States have access to high-quality inpatient care regardless of where they live.

Corresponding Author: Karen E. Joynt, MD, MPH, Department of Medicine, Brigham and Women's Hospital, 75 Francis St, Boston, MA 02115 (kjoynt@partners.org).

Author Contributions: Dr Joynt had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Joynt, Jha.

Acquisition of data: Harris, Jha.

Analysis and interpretation of data: Joynt, Orav.

Drafting of the manuscript: Joynt.

Critical revision of the manuscript for important intellectual content: Harris, Orav, Jha.

Statistical analysis: Joynt, Orav.

Administrative, technical, or material support: Harris, Jha.

Study supervision: Jha.

Disclaimer: The views expressed in this article are solely the opinions of the authors and do not necessarily reflect the official policies of the US Department of Veterans Affairs, Department of Health and Human Services, or the Health Resources and Services Administration, nor does mention of the department or agency names imply endorsement by the US government.

Additional Contributions: Jie Zheng, PhD (Department of Health Policy and Management, Harvard School of Public Health), provided assistance with statistical programming. Dr Zheng received compensation as part of her regular employment.