Association of Centers for Medicare & Medicaid Services Overall Hospital Quality Star Rating With Outcomes in Advanced Laparoscopic Abdominal Surgery

Key Points

Question  Do Centers for Medicare & Medicaid Services high-star hospitals have improved outcomes in advanced laparoscopic abdominal surgery compared with low-star hospitals?

Findings  In this administrative database study, compared with low-star hospitals, high-star hospitals had significantly fewer intensive care unit admissions and lower mean cost. Morbidity at low-star hospitals was higher for colorectal surgery but not bariatric or hiatal hernia surgery, and no significant differences in mortality were found between high-star hospitals and low-star hospitals for any advanced laparoscopic abdominal surgery.

Meaning  High Centers for Medicare & Medicaid Services star rating does not consistently correlate with improved patient outcomes but may represent hospitals with improved resource use.

Importance  The Centers for Medicare & Medicaid Services (CMS) recently released the Overall Hospital Quality Star Rating to help patients compare hospitals based on a 5-star scale. The star rating was designed to assess overall quality of the institution; thus, its validity toward specifically assessing surgical quality is unknown.

Objective  To examine whether CMS high-star hospitals (HSHs) have improved patient outcomes and resource use in advanced laparoscopic abdominal surgery compared with low-star hospitals (LSHs).

Design, Setting, and Participants  Using the University HealthSystem Consortium database (which includes academic centers and their affiliate hospitals) from January 1, 2013, through December 31, 2015, this administrative database observational study compared outcomes of 72 662 advanced laparoscopic abdominal operations between HSHs (4-5 stars) and LSHs (1-2 stars). The star rating includes 57 measures across 7 areas of quality. Patients who underwent advanced laparoscopic abdominal surgery, including bariatric surgery (sleeve gastrectomy, Roux-en-Y gastric bypass), colorectal surgery (colectomy, proctectomy), or hiatal hernia surgery (paraesophageal hernia repair, Nissen fundoplication), were included. Risk adjustment included exclusion of patients with major and extreme severity of illness.

Main Outcomes and Measures  Main outcome measures included serious morbidity, in-hospital mortality, intensive care unit admissions, and cost.

Results  A total of 72 662 advanced laparoscopic abdominal operations were performed in patients at 66 HSHs (n = 38 299; mean [SD] age, 51.26 [15.25] years; 12 096 [31.5%] male and 26 203 [68.4%] female; 28 971 [75.6%] white and 9328 [24.4%] nonwhite) and 78 LSHs (n = 34 363; mean [SD] age, 49.77 [14.77] years; 9902 [28.8%] male and 24 461 [71.2%] female; 21 876 [67.6%] white and 12 487 [32.4%] nonwhite). The HSHs were observed to have fewer intensive care unit admissions (1007 [2.6%] vs 1711 [5.0%], P < .001) and lower mean cost ($7866 vs $8708, P < .001). No significant difference was found in mortality between HSHs and LSHs for any advanced laparoscopic abdominal surgery. No significant difference was found in serious morbidity between HSHs and LSHs for bariatric or hiatal hernia surgery. However, for colorectal surgery, serious morbidity was lower at HSHs compared with LSHs (258 [2.2%] vs 276 [2.9%], P = .002).

Conclusions and Relevance  This study found that HSHs treat fewer ethnic minorities and have similar outcomes as LSHs for advanced laparoscopic abdominal operations. However, HSHs may represent hospitals with improved resource use and cost.

The Centers for Medicare & Medicaid Services (CMS) Overall Hospital Quality Star Rating was released in July 2016.^1,2 This star rating system was designed to help patients and their families compare hospitals objectively by using a standardized rating scale, with a 5-star rating being the best. The Overall Hospital Quality Star Rating was calculated using a weighted mean of 7 categories of quality, including mortality, safety of care, readmission, patient experience, effectiveness of care, timeliness of care, and efficient use of medical imaging. The star rating includes a total of 57 different measures in 7 areas of quality that encompass common medical and surgical conditions treated in hospitals. However, the CMS hospital star rating was not designed to specifically assess quality of surgical care because only 7 of the 57 measures were directly related to outcomes of surgical procedures, such as cardiac, orthopedic, colorectal, and gynecologic surgery. Therefore, the value of using the CMS star rating to assess surgical quality is unknown. The objective of this study was to determine whether CMS high-star hospitals (HSHs) have improved outcomes or resource use in advanced laparoscopic abdominal surgery compared with low-star hospitals (LSHs).

Data were obtained from the University HealthSystem Consortium (UHC) clinical database. The UHC database provides administrative, clinical, and financial inpatient information from academic health centers and affiliated community hospitals in the United States. Reported information includes but is not limited to patient demographic data, morbidity, in-hospital observed and expected mortality, and estimated cost of inpatient care. The UHC database only reports on in-hospital information and does not provide any follow-up data after discharge. The UHC database uses the All Patient Refined Diagnosis Related Group method to group patients based on severity and complexity of comorbidities and complications into 1 of the 4 severity of illness subclasses: minor severity, moderate severity, major severity, or extreme severity. Approval for the use of the database was obtained from the UHC. This study used an administrative database with deidentified patient data. Because this study does not constitute human subjects research, the University of California, Irvine Medical Center Institutional Review Board waived approval.

Analysis of the UHC database discharge data from January 1, 2013, through December 31, 2015, was performed. Patients who underwent advanced laparoscopic abdominal surgery, including bariatric, colorectal, or hiatal hernia surgery, were included in the study. Principal procedure codes of interest were defined using the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) or International Classification of Diseases, 10th Revision, Clinical Modification/Procedure Coding System (ICD-10-CM/PSC). Inclusion criteria were patients who underwent laparoscopic sleeve gastrectomy (ICD-9-CM code 4382; ICD-10-PSC code 0DB64Z3), laparoscopic Roux-en-Y gastric bypass (ICD-9-CM code 4438; ICD-10-PSC code 0D164ZA), laparoscopic colectomy or proctectomy (ICD-9-CM codes 1731, 1732, 1733, 1734, 1735, 1736, 1739, 4581, 4842, and 4851; ICD-10-PSC codes 0DBE4ZZ, 0DTH4ZZ, 0DTF4ZZ, 0DTL4ZZ, 0DTG4ZZ, 0DTN4ZZ, 0DBE4ZZ, 0DTE4ZZ, and 0DTP4ZZ), laparoscopic paraesophageal hiatal hernia repair (ICD-9-CM code 5371; ICD-10-PSC codes 0BQS4ZZ and 0BQR4ZZ), or laparoscopic Nissen fundoplication (ICD-9-CM code 4467; ICD-10-PSC code 0DV44ZZ).

Overall hospital star ratings were obtained directly from the CMS website to compare outcomes of HSHs with LSHs. This star rating system was initially released in July 2016 and later modified in January 2017. The CMS Overall Hospital Quality Star Rating originally represented 64 different measures but currently has 57 measures across 7 areas of quality in common medical and surgical conditions. Only 7 of the 57 measures are directly related to outcomes of surgical procedures. The surgical subspecialties represented in the CMS Overall Hospital Quality Star Rating include only cardiac, orthopedic, colorectal, and gynecologic surgery. The 57 measures are divided into 7 categories of quality. The Overall Hospital Quality Star Rating was calculated using a weighted mean of these 7 categories of quality, which include mortality, safety of care, readmission, patient experience, effectiveness of care, timeliness of care, and efficient use of medical imaging. Data used to calculate the Overall Hospital Quality Star Rating were obtained by the CMS through numerous sources, including the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey, Medicare enrollment and claims data, data submitted by hospitals through the CMS Certification and Survey Provider Enhanced Reporting system, and Centers for Disease Control and Prevention data. In this study, HSHs were defined as hospitals with an overall hospital star rating of 4 or 5 stars, and LSHs were defined as those with an overall hospital star rating of 1 or 2 stars.

Data analyzed included patient characteristics, including age, sex, race, and severity of illness class. The UHC database uses the All Patient Refined Diagnosis Related Group method to group patients on the basis of severity and complexity of comorbidities and complications into the following classes of illness: minor, moderate, major, or extreme severity. For risk adjustment, patients with major or extreme severity of illness classes were excluded from analysis. Main outcome measures included serious morbidity, in-hospital mortality, intensive care unit (ICU) admissions, and cost. Serious morbidity included anastomotic leak, sepsis, bowel obstruction, pneumonia, cerebrovascular accident, respiratory failure, postoperative bleeding, acute renal failure, cardiac complications, deep venous thrombosis, and pulmonary embolism. Data were expressed as mean (SD). GraphPad Prism (GraphPad Inc) statistical software was used for statistical analysis. Proportional differences were analyzed using χ² tests. Mean differences were analyzed using 2-tailed paired t tests. P < .05 (2-sided) was considered to be statistically significant.

A total of 72 662 advanced laparoscopic abdominal operations were performed in patients at 66 HSHs (n = 38 299; mean [SD] age, 51.26 [15.24] years; 12 096 [31.5%] male and 26 203 [68.4%] female; 28 971 [75.6%] white and 9328 [24.4%] nonwhite) and 78 LSHs (n = 34 363; mean [SD] age, 49.77 [14.77] years; 9902 [28.8%] male and 24 461 [71.2%] female; 21 876 [67.6%] white and 12 487 [32.4%] nonwhite). Table 1 gives the demographic data of patients at HSHs vs LSHs who underwent advanced laparoscopic abdominal surgery.

Table 2 gives the demographic data of patients at HSHs vs LSHs who underwent laparoscopic bariatric, colorectal, or hiatal hernia surgery. A total of 42 808 laparoscopic bariatric operations were performed, including 22 024 at HSHs and 20 784 at LSHs. A total of 21 136 laparoscopic colorectal operations were performed, including 11 658 at HSHs and 9478 at LSHs. A total of 8718 laparoscopic hiatal hernia operations were performed, including 4617 at HSHs and 4101 at LSHs. The HSHs consistently treated a lower proportion of nonwhites when compared with LSHs for bariatric surgery (6675 [30.3%] vs 8847 [42.6%], P < .001), colorectal surgery (2025 [17.4%] vs 2756 [29.1%], P < .001), and hiatal hernia surgery (628 [13.6%] vs 884 [21.6%], P < .001).

Table 3 gives the outcomes of patients at HSHs vs LSHs who underwent advanced laparoscopic abdominal surgery. No significant difference in serious morbidity was found between HSHs and LSHs (456 [1.2%] vs 464 [1.3%], P = .06). In-hospital mortality was also similar between groups (22 [0.1%] vs 31 [0.1%], P = .13). The proportion of patients requiring ICU admission was significantly higher at LSHs compared with HSHs (1711 [5.0%] vs 10078 [2.6%], P < .001). Mean (SD) cost was significantly higher at LSHs compared with HSHs ($8708 [$5651] vs $7866 [$4772], P < .001).

Table 4 gives the outcomes of the subset analysis of patients at HSHs vs LSHs who underwent laparoscopic bariatric surgery, colorectal surgery, or hiatal hernia surgery. No significant differences in serious morbidity were found between HSHs and LSHs in bariatric surgery (124 [0.6%] vs 126 [0.6%], P = .60) and hiatal hernia surgery (74 [1.6%] vs 62 [1.5%], P = .80). However, HSHs had lower serious morbidity in colorectal surgery compared with LSHs (258 [2.2%] vs 276 [2.9%], P = .002). In-hospital mortality was similar between HSHs and LSHs in bariatric surgery (3 [0.01%] vs 7 [0.03%], P = .29), colorectal surgery (9 [0.1%] vs 13 [0.1%], P = .26), and hiatal hernia surgery (10 [0.2%] vs 11 [0.3%], P = .79). In addition, the mortality index (observed/expected) was less than 1 for HSHs and LSHs. The proportion of patients requiring ICU admission was significantly higher at LSHs compared with HSHs in bariatric surgery (788 [3.8%] vs 297 [1.4%], P < .001), colorectal surgery (502 [5.3%] vs 474 [4.1%], P < .001), and hiatal hernia surgery (420 [10.2%] vs 235 [5.1%], P < .001). Mean (SD) cost was significantly higher at LSHs compared with HSHs for bariatric surgery ($8216 [$5172] vs $7420 [$4271], P < .001), colorectal surgery ($9947 [$6318] vs $8866 [$5281], P < .001), and hiatal hernia surgery ($8273 [$5829] vs $7475 [$5281], P < .001).

The recently released CMS Overall Hospital Quality Star Rating was designed to help patients make an informed decision about where to go for care of common medical and surgical conditions. However, the accuracy of this overall hospital star rating in predicting improved surgical care for minimally invasive abdominal surgery is unknown. In this study, we used the UHC database to determine whether there is a correlation between a hospital’s star rating and outcomes of advanced laparoscopic abdominal operations. We found that HSHs tend to treat a lower proportion of ethnic minorities. Despite this finding, HSHs do not consistently have improved outcomes in advanced laparoscopic abdominal operations, specifically in laparoscopic bariatric, colorectal, or hiatal hernia surgery. However, HSHs have improved resource use, with fewer ICU admissions and lower cost.

Our findings indicate that the recent CMS star rating is a poor surrogate for quality of surgical care for common advanced laparoscopic abdominal operations. The released CMS star rating can be confusing to consumers who likely will use the star rating to identify hospitals for their surgical care. We found that LSHs consistently treat a higher proportion of ethnic minorities. This finding is important because previous studies^3,4 have found that ethnic minorities tend to have worse outcomes in similar medical and surgical conditions. In 1 study,³ black individuals were found to have higher operative mortality across a wide range of surgical procedures. Despite the higher proportion of ethnic minorities at LSHs, no significant differences in outcomes were found between HSHs and LSHs in common advanced laparoscopic operations. The only significant difference between the 2 groups was higher serious morbidity for colorectal surgery at LSHs compared with HSHs. Only 1 prior study⁵ has examined the association between CMS hospital star rating and patient outcomes. Wang et al⁵ examined mortality related to medical conditions (acute myocardial infarction, pneumonia, and heart failure) in the Medicare population and found that LSHs had higher mortality and readmissions compared with HSHs. That study did not examine surgical conditions specifically. In addition, the study was performed using the prior CMS hospital star rating, which was dependent solely on patient experience based on the HCAHPS survey. The most recent CMS star rating system released in July 2016 includes patient experience through the HCAHPS survey and 6 additional quality measures, including mortality, safety of care, readmission rate, effectiveness of care, timeliness of care, and efficient use of medical imaging.

A criticism of the CMS star rating is that the chosen composite measures may be too broad and may be more reflective of medical rather than surgical quality. Of the 57 quality measures, only 7 measures were related to surgical conditions. Therefore, the use of the CMS star rating can be misleading for patients who use this rating system to choose a hospital for their surgical care. In our study, we found that HSHs were associated with reduced serious morbidity, specifically for colorectal surgery. This finding may be because 1 of the 7 surgical quality measures specifically examined surgical site infection in colorectal surgery.¹ Use of more targeted surgical quality measures may improve the CMS star rating and make it more applicable for surgical disciplines.^6,7 Although higher overall hospital star rating was not associated with improved outcome, we found that HSHs had fewer ICU admissions and lower cost.

The UHC database is limited to only academic centers and their affiliated hospitals. Inferences from our findings cannot be made about the outcomes at community hospitals. Despite this limitation, the UHC database was selected for this study because it allows selection of specific hospitals by hospital code. To our knowledge, no other broad, nationwide database is available that allows selection of individual hospitals. The UHC database is limited to in-hospital data only and does not contain follow-up data or any data after discharge. Any complications or deaths that occur after discharge would not be captured in this database. Therefore, the reported morbidity and mortality are likely to be underestimated. Another limitation is that risk adjustment using specific comorbidities at the patient level is not possible. Last, the present study reflects analysis of outcomes for advanced laparoscopic abdominal operations only, and the findings do not apply to other medical or surgical conditions. Despite these limitations, our study is the first, to our knowledge, to analyze the association between the CMS overall hospital star rating and surgical outcomes in advanced laparoscopic operations.

In this study of advanced laparoscopic abdominal operations, we found that HSHs do not consistently have improved surgical outcomes compared with LSHs. However, HSHs have improved resource use and cost. These findings should be considered in the context that HSHs consistently treated fewer ethnic minorities than LSHs. On the basis of these findings, the CMS overall hospital star rating may be misleading for patients and their families in selecting a hospital for their minimally invasive abdominal surgical care.

Corresponding Author: Ninh T. Nguyen, MD, Department of Surgery, University of California, Irvine Medical Center, 333 City Bldg W, Ste 1600, Orange, CA 92868 (ninhn@uci.edu).

Accepted for Publication: April 16, 2017.

Published Online: July 5, 2017. doi:10.1001/jamasurg.2017.2212

Author Contributions: Drs Koh and Nguyen had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: All authors.

Acquisition, analysis, or interpretation of data: Koh, Inaba.

Drafting of the manuscript: Koh.

Critical revision of the manuscript for important intellectual content: All authors.

Administrative, technical, or material support: Inaba, Sujatha-Bhaskar, Nguyen.

Study supervision: Nguyen.

Conflict of Interest Disclosures: None reported.

Meeting Presentation: This work was presented at the 88th Annual Meeting of the Pacific Coast Surgical Association; February 18, 2017; Indian Wells, California.