[Skip to Content]
[Skip to Content Landing]
Figure.
Mean No. of Hospital Quality Measures Reported (of 62 Total Possible Measures) by Hospital Typea
Mean No. of Hospital Quality Measures Reported (of 62 Total Possible Measures) by Hospital Typea

aError bars represent 95% CIs. Includes hospitals that met the reporting threshold of at least 3 measures within at least 1 Outcome group (mortality, readmissions, safety). Using the Kruskal-Wallis test, P<.001 for all pairwise comparisons except between critical access and specialty hospitals (P = .96).

Table.  
Hospital Characteristics by Centers for Medicare & Medicaid Services (CMS) Overall Hospital Quality Star Ratings
Hospital Characteristics by Centers for Medicare & Medicaid Services (CMS) Overall Hospital Quality Star Ratings
1.
QualityNet.  Comprehensive Methodology Report, v2.0. 2016; https://www.qualitynet.org/dcs/ContentServer?c=Page&pagename=QnetPublic%2FPage%2FQnetTier3&cid=1228775957165. August 16, 2016.
2.
Bilimoria  KY, Barnard  C.  The new CMS Hospital Quality Star Ratings: the stars are not aligned.  JAMA. 2016;316(17):1761-1762.PubMedGoogle ScholarCrossref
3.
QualityNet.  Statistical Analysis System (SAS) package. https://www.qualitynet.org/dcs/ContentServer?c=Page&pagename=QnetPublic%2FPage%2FQnetTier3&cid=1228775958130. Accessed 8/14/2016.
4.
US Government Publishing Office.  Federal Register 2015: rules and regulations. https://www.gpo.gov/fdsys/pkg/FR-2015-08-17/pdf/2015-19049.pdf. Accessed September 22, 2016.
Research Letter
May 16, 2017

Associations Between Hospital Characteristics, Measure Reporting, and the Centers for Medicare & Medicaid Services Overall Hospital Quality Star Ratings

Author Affiliations
  • 1Surgical Outcomes and Quality Improvement Center (SOQIC), Northwestern University Feinberg School of Medicine, Chicago, Illinois
  • 2Division of Quality, Northwestern Memorial HealthCare, Chicago, Illinois
JAMA. 2017;317(19):2015-2017. doi:10.1001/jama.2017.3148

The Centers for Medicare & Medicaid Services (CMS) introduced the Overall Hospital Quality Star Ratings in July 2016 to provide consumers a summary rating of hospital performance using measures already reported to CMS.1 Hospitals achieve 1 to 5 stars based on 62 measures across 7 weighted measure groups relating to health care quality: mortality, readmissions, safety, patient experience, efficient use of imaging, effectiveness of care, and timeliness of care.1 To receive a star rating, hospitals must report at least 3 of the 7 measure groups and at least 1 Outcome group (mortality, readmissions, or safety). Within each measure group, at least 3 individual measures must be reported for inclusion of that group. If a hospital does not submit enough measures for a group to be included, the weight from that group is redistributed to the other groups for which the hospital has sufficient data.

Concerns have been raised regarding the measures, methodology, and potential bias toward or against certain hospital types.2 To assess these concerns, we evaluated associations between hospital characteristics, number and types of measures reported, and the star ratings.

Methods

This study was deemed nonhuman subjects research by the Northwestern University institutional review board. Data from the CMS Overall Hospital Quality Star Ratings database (obtained July 27, 2016), American Hospital Association 2014 Annual Survey of Hospitals, and CMS Fiscal Year 2015 Payment Update Impact File were merged.1,3 We examined associations between hospital characteristics (Table), number of individual measures reported, number of group categories (Outcome or other) reported, and the star ratings (4-5 stars [high] vs 1-3 stars [low]) using χ2 analysis and Kruskal-Wallis tests. Two-sided P values less than .05 were used to indicate statistical significance. Analyses were performed using Statistical Analysis Software (SAS Institute), version 9.4.

Results

Of 3591 hospitals receiving a star rating, 4 or 5 stars were awarded to 15.8% of major teaching hospitals, 18.8% of other teaching hospitals, 30.2% of community hospitals, 33.3% of critical access hospitals, and 87.3% of specialty hospitals (P < .001 for all compared with specialty hospitals) (Table). Smaller hospitals more frequently achieved a high star rating compared with larger hospitals (No. of beds: <100, 34.0%; 100-299, 24.3%; 300-499, 24.2%; ≥500, 19.4%; P < .001). Hospitals with lower disproportionate share payments achieved a high star rating more frequently (Q1, 42.8%; Q2, 29.7%; Q3, 23.1%; Q4, 11.6%; P < .001). Specialty hospitals comprised 2.2% of hospitals but accounted for 41.1% of 5-star hospitals.

Hospitals reported a mean of 46.6 (75.2%) of 62 possible measures. Major teaching (mean, 56.9), other teaching (mean, 52.1), and community hospitals (mean, 49.5) reported approximately twice as many measures as critical access (mean, 25.9) and specialty hospitals (mean, 26.5; P < .001) (Figure). Hospitals reporting more Outcome measures less frequently received a high star rating: high star ratings were given to 46.9% of hospitals reporting the minimum threshold of 1 Outcome measure group, 31.2% of hospitals reporting 2 Outcome measure groups, and 25.5% of hospitals reporting all 3 Outcome measure groups (P < .001) (Table). Star ratings were based on the minimum eligibility threshold of only 3 of 7 measure groups for 49.4% of specialty and 33.3% of critical access hospitals; whereas, 94.3% of major teaching, 80.8% of other teaching, and 96% of community hospitals reported all 7 measure groups.

Discussion

Hospitals less frequently received a high star rating if they were larger, academic hospitals or cared for a higher proportion of disproportionate share patients. Specialty and critical access hospitals more frequently earned high star ratings compared with acute care hospitals. Critical access hospitals and some specialty hospitals (ie, certain cancer centers) are exempt from reporting-based payment incentives through the CMS Inpatient Quality Reporting system and may not collect many measures used in the star ratings.4 Consequently, specialty and critical access hospitals reported systematically fewer measures. Although hospital type influenced the number and type of measures reported, the study was limited by the inability to determine whether differences in individual measure reporting by hospital type explained the differences in star ratings.

Because the measures used as the basis for calculating the star ratings differed by hospital type, failure to account for these differences may limit the utility of the star ratings, particularly when comparing different hospital types.

Section Editor: Jody W. Zylke, MD, Deputy Editor.
Back to top
Article Information

Accepted for Publication: March 8, 2017.

Corresponding Author: Karl Y. Bilimoria, MD, MS, Surgical Outcomes and Quality Improvement Center (SOQIC), Department of Surgery and Center for Healthcare Studies, Northwestern University Feinberg School of Medicine, 633 St Clair St, 20th Floor, Chicago, IL 60611 (k-bilimoria@northwestern.edu).

Author Contributions: Drs Bilimoria and DeLancey had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: DeLancey, Barnard, Bilimoria.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: DeLancey, Bilimoria.

Critical revision of the manuscript for important intellectual content: Softcheck, Chung, Barnard, Dahlke, Bilimoria.

Statistical analysis: DeLancey, Softcheck, Bilimoria.

Obtained funding: Bilimoria.

Administrative, technical, or material support: Barnard, Dahlke.

Supervision: Barnard, Bilimoria.

Feedback and debate: Chung.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Bilimoria reports having received support from the National Institutes of Health, Agency for Healthcare Research and Quality, American Board of Surgery, American College of Surgeons, Accreditation Council for Graduate Medical Education, Health Care Services Corporation, Blue Cross Blue Shield of Illinois, Mallinckrodt, and Northwestern University. No other disclosures were reported.

Funding/Support: This work is supported by grant R21HS021857 from the Agency for Healthcare Research and Quality (Dr Bilimoria).

Role of Funder/Sponsor: The funder had no role in design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

References
1.
QualityNet.  Comprehensive Methodology Report, v2.0. 2016; https://www.qualitynet.org/dcs/ContentServer?c=Page&pagename=QnetPublic%2FPage%2FQnetTier3&cid=1228775957165. August 16, 2016.
2.
Bilimoria  KY, Barnard  C.  The new CMS Hospital Quality Star Ratings: the stars are not aligned.  JAMA. 2016;316(17):1761-1762.PubMedGoogle ScholarCrossref
3.
QualityNet.  Statistical Analysis System (SAS) package. https://www.qualitynet.org/dcs/ContentServer?c=Page&pagename=QnetPublic%2FPage%2FQnetTier3&cid=1228775958130. Accessed 8/14/2016.
4.
US Government Publishing Office.  Federal Register 2015: rules and regulations. https://www.gpo.gov/fdsys/pkg/FR-2015-08-17/pdf/2015-19049.pdf. Accessed September 22, 2016.
×