The point estimates are in black and the 95% CIs in yellow.
Lucas DJ, Ejaz A, Bischof DA, Schneider EB, Pawlik TM. Variation in Readmission by Hospital After Colorectal Cancer Surgery. JAMA Surg. 2014;149(12):1272-1277. doi:10.1001/jamasurg.2014.988
Hospital readmission after colorectal surgery is common, with reported 30-day readmission rates ranging from 10% to 14%. Readmission has become a major hospital quality metric, but it is unclear whether there is much difference in readmission among hospitals after appropriate risk adjustment.
To assess the variability in risk-adjusted readmission rates among hospitals after colorectal surgery.
Design, Setting, and Participants
We performed a hierarchical multivariable logistic regression analysis of observational data obtained from the Surveillance, Epidemiology, and End Results–Medicare linked database, a nationally representative cancer registry. We studied 44 822 patients who underwent colorectal resection for cancer at 1401 US hospitals from January 1, 1997, through December 31, 2002.
Main Outcomes and Measures
Variation in risk-adjusted 30-day readmission among hospitals.
The median age of the study patients was 78 years (interquartile range [IQR], 72-83 years). The overall 30-day readmission rate was 12.3% (n = 5502). Looking at hospitals that performed at least 5 operations annually, we found marked variation in raw readmission rates, with a range of 0% to 41.2% (IQR, 9.5%-14.8%). However, after adjusting for patient characteristics, comorbidities, and operation types in a hierarchical model, no significant variability was found in readmission rates among hospitals, with a range of 11.3% to 13.2% (IQR, 12.1%-12.4%). Furthermore, the 95% CI for hospital-specific readmission overlapped the overall mean at every hospital.
Conclusions and Relevance
Little risk-adjusted variation exists in hospital readmission rates after colorectal surgery. The use of readmission rates as a high-stakes quality measure for payment adjustment or public reporting across surgical specialties should proceed cautiously and must include appropriate risk adjustment.
Quiz Ref IDHospital readmission after surgery is common, ranging from 5% to 16% by specialty.1 After colorectal surgery, the rate of readmission within 30 days of discharge has been reported to be approximately 10% to 14%.1- 4 Readmission often represents an adverse patient event, results in increased cost of care, and sometimes serves as an indicator of underlying poor care.5 The Centers for Medicare & Medicaid Services (CMS) emphasized reducing unplanned readmissions and initiated the Hospital Readmissions Reduction Program in 2012. This program assesses up to a 3% penalty against a hospital’s overall reimbursement if there is “excess” readmission for certain diagnoses. The conditions that currently fall under the auspices of the program include acute myocardial infarction, heart failure, pneumonia, chronic obstructive pulmonary disorder, and total hip and knee arthroplasty.6 The list of monitored conditions is likely to increase in the near future and encompass more surgical procedures. By determining variation in readmission rates among hospitals, high outlier hospitals can be identified and targeted to reduce readmission. We sought to determine whether readmission rates after colorectal surgery vary among hospitals and identify potential risk factors for readmission.
This study was approved by the Johns Hopkins University School of Medicine Institutional Review Board. Informed consent was waived for this retrospective, minimal-risk analysis of deidentified patient information from a national database. A retrospective analysis of readmission after colorectal surgery was performed with the Surveillance, Epidemiology, and End Results (SEER)–Medicare linked database.7 Patients who underwent a colectomy or proctectomy for colorectal cancer from January 1, 1997, through December 31, 2002, were selected. Patients whose eligibility for Medicare was due to end-stage renal disease or disability were excluded from the study.
Quiz Ref IDPatient demographics, comorbidities, and operative details were obtained, along with dates of admission, discharge, and readmission from Medicare records. Comorbidities were defined according to the Charlson Comorbidity Index (Table 1),8 excluding metastatic cancer (which has poor sensitivity in Medicare databases and is not recommended for use) and AIDS (no patient in the study had AIDS). Although this index was originally designed to predict 1-year mortality for hospitalized patients, it has been validated for use in perioperative outcomes research.9 Readmission was defined as admission to any acute care hospital for any reason within 30 days of discharge after initial resection. Readmission to the same hospital and other hospitals is well captured in Medicare data because the CMS tracks all billing records generated for care. Hospitals were identified using pseudoidentifiers based on the Unique Physician Identification Number in the Medicare database.
Continuous variables are presented as medians with interquartile ranges (IQRs), where appropriate. Categorical variables are presented as whole numbers and percentages. Associations between readmission and demographics, operation type, and comorbidities were calculated using univariable and multivariable hierarchical logistic regression models after calculating each hospital’s individual risk-adjusted readmission rate. Variation among hospitals was assessed by comparing these individual risk-adjusted readmission odds ratios (ORs), similar to the models currently used for hospital profiling by the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP)10 and the CMS.11 This model adjusts for patient factors and filters out statistical noise frequently seen in outlier values at hospitals with low case volumes. Adjustment variables within the model were applied as previously described elsewhere.1,12 Because the focus of the study was perioperative readmission, specific tumor variables, such as stage and histologic type, were not included in the analysis because they have not improved modeling in this context.13
Some of the hospital-level analyses looked only at those hospitals with an annual case volume of at least 5 operations. There were a large number of hospitals that contributed only a few cases (872 hospitals with <5 cases annually), but these hospitals only accrued 10.2% of the total number of operations. Hierarchical models draw the predicted readmission rates at these small hospitals strongly toward the overall mean, which is essentially uninformative. Because data for these hospitals can distort the overall distribution of risk-adjusted readmission rates (by concentrating them at the mean), they were omitted from some analyses, which is clearly specified.
All statistical analyses were performed with STATA statistical software, version 13 (StataCorp), using P < .05 for 2-tailed tests as a cutoff for statistical significance. No data were missing.
We identified 44 822 patients at 1401 hospitals eligible for this study using the SEER-Medicare database. The mean number of patients per hospital was 32, with a median of 9 patients (IQR, 2-40 patients). A total of 289 hospitals contributed at least 50 patients, whereas 126 hospitals had 100 patients or more. Overall, the 30-day readmission rate among all hospitals was 12.3% (n = 5502).
The median age of the cohort was 78 years (IQR, 72-83 years), whereas most patients were non-Hispanic white (n = 39 032 [87.1%]) and female (n = 24 755 [55.2%]) (Table 1). The most common comorbidities in the cohort were chronic obstructive pulmonary disease (n = 6864 [15.3%]), diabetes mellitus (n = 5951 [13.3%]), and congestive heart failure (n = 4791 [10.7%]). At the time of surgery, 37 228 patients (83.1%) underwent a colectomy, with the most common operation being a right colectomy (n = 20 881 [46.6%]); 7534 patients (16.8%) underwent a proctectomy.
Multiple risk factors were associated with readmission on hierarchical logistic regression modeling (Table 2). On adjusted analysis, risk of readmission increased progressively with age (ORs progressively increasing with each subsequent decade, all P ≤ .05) and among those patients of black race (OR, 1.24; 95% CI, 1.11-1.39; P < .001). Operations most associated with readmission were total abdominal colectomy (OR, 1.82; 95% CI, 1.48-2.22; P < .001) and abdominoperineal resection (OR, 1.19; 95% CI, 1.05-1.35; P = .008) compared with patients undergoing a right colectomy. Patients undergoing sigmoid colectomy were at decreased risk of odds for readmission (OR, 0.88; 95% CI, 0.81-0.96; P = .003). Several comorbidities were associated with increased readmission, including congestive heart failure (OR, 1.55; 95% CI, 1.43-1.68; P < .001), chronic obstructive pulmonary disease (OR, 1.22; 95% CI, 1.14-1.32; P < .001), and cerebrovascular disease (OR, 1.24; 95% CI, 1.05-1.47; P = .01).
There was marked variation in the unadjusted risk of readmission among hospitals. Among hospitals that contributed at least 5 cases per year, the 30-day readmission rate ranged from 0% to 41.2%, with an IQR of 9.5% to 14.8% (Figure 1). Quiz Ref IDIn contrast, this high variation in readmission among hospitals markedly decreased after adjusting for comorbidities within a hierarchical model. After risk adjustment, no hospital, including those with fewer than 5 cases annually, had a readmission OR significantly different than the overall mean (Figure 2). After applying these adjusted ORs to the group mean for readmission (12.3%), the resulting estimated 30-day readmission among these hospitals had a much narrower distribution, ranging from 11.3% to 13.2% (IQR, 12.1%-12.4%) among hospitals with at least 5 cases per year (Figure 3).
Annual cases per hospital were calculated. Patients were divided into quartiles by annual hospital volume. These volume quartiles were then added to the prior hierarchical model. Quiz Ref IDHospitals in the first quartile (lowest volume), which corresponded to fewer than 10 cases annually, had somewhat higher readmission rates compared with hospitals in the fourth quartile (highest volume) (OR, 1.16; 95% CI, 1.07-1.26; P < .001). However, hospitals in the second and third volume quartiles had equivalent readmission rates as the fourth quartile (second quartile: OR, 1.00; 95% CI, 0.92-1.10; P = .84; third quartile: OR, 1.00; 95% CI, 0.92-1.09; P = 89).
The emphasis by the CMS on reducing unplanned hospital readmissions has generated increasing interest in readmission. Given that readmission is likely to be included in the public reporting of certain hospital-based metrics, any variability among hospitals has heightened implications. Hospitals with high readmission rates are also now penalized monetarily; therefore, identifying possible risk factors responsible for high readmission rates is paramount. In addition, reducing readmission rates among high outlier hospitals may help improve overall quality of care. Of interest, there is much debate as to whether variability in readmission rates actually exists among hospitals and, if present, how the extensive the variability may be. This article is important because it reveals that significant variation exists with regard to readmission after colorectal surgery when examining raw readmission rates. In fact, readmission among hospitals varied widely from 0% to 41.2% among hospitals with at least 5 cases per year. More important, however, was the finding that this variation largely disappeared after risk adjustment within a hierarchical logistic regression model. After accounting for patient- and hospital-level factors using hierarchical adjustment models, there was minimal variation in the estimated 30-day readmission among hospitals (11.3%-13.2%). These data have important implications because they strongly suggest that minimal risk-adjusted variation exists in hospital readmission rates after colorectal surgery.
Although multivariable analyses can adjust for different patient-level risk factors using standard logistic regression techniques, hierarchical models better account for the statistical uncertainty related to low case volumes seen at the hospital level. By applying Bayesian methods that take information from the overall group of hospitals and apply that information to individual small hospitals, hierarchical modeling can yield more accurate predictions of future performance with much narrower CIs.14 This approach is especially important for low-volume hospitals because the low volumes at these institutions can lead to wide CIs when using standard logistic modeling. In a prior study, Zheng et al15 found lower than expected variation in mortality rates after colorectal cancer in California hospitals after applying a similar hierarchical risk-adjusted model. Similarly, our hierarchical risk-adjusted model revealed a mean readmission rate of 11.3% to 13.2% across all hospitals, a much narrower estimate than the unadjusted raw rates. Furthermore, all hospitals had 95% CIs that overlapped the group mean. In essence, we found that no hospital had a statistically distinguishable readmission rate in the hierarchical risk-adjusted model. Tsai et al16 examined readmission after 6 major operation types (coronary artery bypass grafting, pulmonary lobectomy, endovascular repair of abdominal aortic aneurysm, open repair of abdominal aortic aneurysm, colectomy, and hip replacement) using Medicare data. In this study, the authors reported substantial variation in readmission rates among hospitals, with higher-volume hospitals and those hospitals with higher surgical quality scores having lower readmission rates. Data from the current study differed from the findings of the study by Tsai et al16 in several important ways. Tsai et al did not formally test whether the variation seen among hospitals was statistically significant but instead merely described the variation in readmission by hospital and subjectively judged it to be meaningful. Furthermore, they used nonhierarchical logistic regression modeling, which is known to overestimate variation, compared with hierarchical models, which have become the standard.14 Similar to their study, we also noted a volume-outcome association with readmission but found it to be modest. Compared with patients treated at hospitals in the highest volume quartile, patients in the first quartile (hospitals with <10 cases annually) had 16% higher odds of readmission, and patients in the second and third quartiles had identical readmission rates as the fourth quartile. Although higher-volume hospitals as a group may have a somewhat lower readmission rate than lower-volume hospitals, this pools volume among hospitals to improve statistical power; it is much more difficult to reveal that any one particular hospital has a different readmission rate than another.
Hierarchical modeling for hospital profiling has been endorsed by the Committee of Presidents of Statistical Societies, the CMS, and the ACS NSQIP.10,11 The CMS Hospital Readmission Reduction Program uses a robust hierarchical model that adjusts for comorbidities with a very granular level of detail.6,17 However, penalties are assessed based on the point estimate of the adjusted readmission rate without taking into account the 95% CI of the estimate, with a penalty for all hospitals above the mean. On the basis of this method, 50% of hospitals would receive a penalty for “excess” readmission, even if the calculated CI includes the overall mean. Despite the use of a robust hierarchical model to determine a more accurate readmission rate, the enforcement of the policy seems statistically questionable because it ignores the uncertainty that is defined by the CIs. Furthermore, this method stands in direct contrast to the public reporting of these same measures on the CMS Hospital Compare website, where hospitals are only flagged if the 95% CI for their risk-adjusted readmission rate is above the national mean. Although one could argue for a narrower CI than 95% when making determinations regarding which hospitals have poor performance, it is problematic to not use a CI at all, especially given the substantial financial penalties involved.
A potential method to help decrease the amount of uncertainty surrounding calculated readmission rates among hospitals would be to increase the number of patients included in the calculation. The ACS NSQIP reports readmission across entire surgical specialties rather than for specific procedure types, with a requirement of at least 1680 cases annually.10 Although this may not be an issue with common ailments, such as acute myocardial infarction or pneumonia, low surgical volume in any one particular operation would result in an imprecise calculation of readmission.18 As an alternative, risk-adjusted readmission rates could be calculated for a surgical subspecialty or even across an entire department, rather than focusing on single operations, such as colectomy or coronary artery bypass graft. This approach would allow a broader evaluation of the hospital staff and the surgical team, which all contribute to differing rates in readmission.1
Quiz Ref IDAfter the application of a hierarchical risk-adjusted model, no statistically significant variability was found in readmission rates after surgery for colorectal cancer among hospitals. The use of risk-adjusted readmission rates as a high-stakes quality measure for payment adjustment or public reporting across surgical specialties should proceed cautiously.
Accepted for Publication: February 26, 2014.
Corresponding Author: Timothy M. Pawlik, MD, MPH, PhD, Department of Surgery, Johns Hopkins University School of Medicine, 600 N Wolfe St, Blalock 688, Baltimore, MD 21287 (email@example.com).
Published Online: October 22, 2014. doi:10.1001/jamasurg.2014.988.
Author Contributions: Drs Pawlik and Lucas had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Lucas, Schneider, Pawlik.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Lucas.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Lucas, Ejaz, Schneider, Pawlik.
Administrative, technical, or material support: Pawlik.
Study supervision: Schneider, Pawlik.
Conflict of Interest Disclosures: None reported.
Disclaimer: The views expressed in this article are those of the authors and do not represent the official policy of the US Navy, US Department of Defense, or US government.