JAMA Health Forum – Health Policy, Health Care Reform, Health Affairs | JAMA Health Forum | JAMA Network
[Skip to Navigation]
Sign In
Figure 1.  Association Between Simulated Star Ratings for Enrollees with High vs Low Socioeconomic Status (SES) and for White vs Racial/Ethnic Minority Enrollees in the Same Medicare Advantage Contract
Association Between Simulated Star Ratings for Enrollees with High vs Low Socioeconomic Status (SES) and for White vs Racial/Ethnic Minority Enrollees in the Same Medicare Advantage Contract

Each point represents a contract and plots the calculated star rating for each group. Dots above the 45-degree line indicate that either the high SES or the White enrollee stratified star ratings are higher than that of the comparison group. The US Centers for Medicare & Medicaid Services rounds star ratings to the nearest 0.5 increment. Dark blue dots represent that after rounding the star rating for each group would be the same. Orange dots represent that after rounding the stratified star ratings would be different.

Figure 2.  Socioeconomic and Racial/Ethnic Disparity in Simulated Star Rating by Decile of Low Socioeconomic Status (SES) or Race/Ethnicity Within the Medicare Advantage Contract
Socioeconomic and Racial/Ethnic Disparity in Simulated Star Rating by Decile of Low Socioeconomic Status (SES) or Race/Ethnicity Within the Medicare Advantage Contract

Each point represents the disparity in star rating between each group (low SES and high SES or Black or Hispanic and White) by decile of SES or race/ethnicity concentration. Points above the 0 line indicate that the enrollees with low SES or Black or Hispanic enrollees in a contract perform worse in star-rating calculations than the comparison groups. If the 95% CIs do not cross the line at 0, then there is a statistically significant disparity at the .05 level. The cut points for each decile are included in eTable 2 in the Supplement.

Figure 3.  Socioeconomic and Racial/Ethnic Disparity in Simulated Star Ratings by US Centers for Medicare & Medicaid Services’ Published Contract-level Star Rating
Socioeconomic and Racial/Ethnic Disparity in Simulated Star Ratings by US Centers for Medicare & Medicaid Services’ Published Contract-level Star Rating

The y-axis in each panel represents either the average difference in stratified star rating between enrollees with low socioeconomic status (SES) and high SES or between White enrollees and Black or Hispanic enrollees. The x-axis represents the contract’s official star rating from 2016. Points above the 0 line indicate that the contracts perform worse for enrollees with low SES or Black and Hispanic enrollees. If the 95% CIs do not cross the line at 0, then there is a statistically significant disparity at the .05 level. For enrollees with low SES, the 4-star and 4.5- to 5-star–level disparities are significantly higher than the 2.5-star and 3-star groups (P < .05). For Black enrollees, the 4-star and 4.5- to 5-star–level disparities are significantly higher than the 3-star group (P < .05). For Hispanic enrollees, the 4-star and 4.5- to 5-star–level disparities are significantly higher than the 2.5-star and 3-star groups (P < .05).

Table 1.  Demographic and Contract Characteristics of Study Population by Socioeconomic Status (SES) and Race/Ethnicitya
Demographic and Contract Characteristics of Study Population by Socioeconomic Status (SES) and Race/Ethnicitya
Table 2.  Simulated Contract Star Ratings Stratified by Socioeconomic Status (SES) and Race/Ethnicitya
Simulated Contract Star Ratings Stratified by Socioeconomic Status (SES) and Race/Ethnicitya
1.
Neuman  P, Jacobson  GA.  Medicare Advantage checkup.   N Engl J Med. 2018;379(22):2163-2172. doi:10.1056/NEJMhpr1804089 PubMedGoogle ScholarCrossref
2.
Meyers  DJ, Johnston  KJ.  The Growing Importance of Medicare Advantage in Health Policy and Health Services Research.   JAMA Health Forum. 2021;2(3):e210235. doi:10.1001/jamahealthforum.2021.0235Google Scholar
3.
Trivedi  AN, Zaslavsky  AM, Schneider  EC, Ayanian  JZ.  Relationship between quality of care and racial disparities in Medicare health plans.   JAMA. 2006;296(16):1998-2004. doi:10.1001/jama.296.16.1998 PubMedGoogle ScholarCrossref
4.
Weech-Maldonado  R, Elliott  MN, Adams  JL,  et al.  Do racial/ethnic disparities in quality and patient experience within Medicare plans generalize across measures and racial/ethnic groups?   Health Serv Res. 2015;50(6):1829-1849. doi:10.1111/1475-6773.12297 PubMedGoogle ScholarCrossref
5.
Li  Y, Cen  X, Cai  X, Thirukumaran  CP, Zhou  J, Glance  LG.  Medicare Advantage associated with more racial disparity than traditional Medicare for hospital readmissions.   Health Aff (Millwood). 2017;36(7):1328-1335. doi:10.1377/hlthaff.2016.1344 PubMedGoogle ScholarCrossref
6.
Martino  SC, Elliott  MN, Hambarsoomian  K,  et al.  Racial/ethnic disparities in Medicare beneficiaries’ care coordination experiences.   Med Care. 2016;54(8):765-771. doi:10.1097/MLR.0000000000000556 PubMedGoogle ScholarCrossref
7.
Ayanian  JZ, Landon  BE, Zaslavsky  AM, Newhouse  JP.  Racial and ethnic differences in use of mammography between Medicare Advantage and traditional Medicare.   J Natl Cancer Inst. 2013;105(24):1891-1896. doi:10.1093/jnci/djt333 PubMedGoogle ScholarCrossref
8.
Elliott  MN, Haviland  AM, Orr  N, Hambarsoomian  K, Cleary  PD.  How do the experiences of Medicare beneficiary subgroups differ between managed care and original Medicare?   Health Serv Res. 2011;46(4):1039-1058. doi:10.1111/j.1475-6773.2011.01245.x PubMedGoogle ScholarCrossref
9.
Rivera-Hernandez  M, Leyva  B, Keohane  LM, Trivedi  AN.  Quality of care for White and Hispanic Medicare Advantage enrollees in the United States and Puerto Rico.   JAMA Intern Med. 2016;176(6):787-794. doi:10.1001/jamainternmed.2016.0267 PubMedGoogle ScholarCrossref
10.
Ayanian  JZ, Landon  BE, Newhouse  JP, Zaslavsky  AM.  Racial and ethnic disparities among enrollees in Medicare Advantage plans.   N Engl J Med. 2014;371(24):2288-2297. doi:10.1056/NEJMsa1407273 PubMedGoogle ScholarCrossref
11.
Chou  AF, Brown  AF, Jensen  RE, Shih  S, Pawlson  G, Scholle  SH.  Gender and racial disparities in the management of diabetes mellitus among Medicare patients.   Womens Health Issues. 2007;17(3):150-161. doi:10.1016/j.whi.2007.03.003 PubMedGoogle ScholarCrossref
12.
Abraham  JM, Marmor  S, Knutson  D, Zeglin  J, Virnig  B.  Variation in diabetes care quality among Medicare Advantage plans: understanding the role of case mix.   Am J Med Qual. 2012;27(5):377-382. doi:10.1177/1062860611428529 PubMedGoogle ScholarCrossref
13.
Breslau  J, Elliott  MN, Haviland  AM,  et al.  Racial and ethnic differences in the attainment of behavioral health quality measures in Medicare Advantage plans.   Health Aff (Millwood). 2018;37(10):1685-1692. doi:10.1377/hlthaff.2018.0655 PubMedGoogle ScholarCrossref
14.
Sorbero  ME, Paddock  SM, Damberg  CL,  et al.  Adjusting Medicare Advantage star ratings for socioeconomic status and disability.   Am J Manag Care. 2018;24(9):e285-e291.PubMedGoogle Scholar
15.
Konetzka  RT, Grabowski  DC, Perraillon  MC, Werner  RM.  Nursing home 5-star rating system exacerbates disparities in quality, by payer source.   Health Aff (Millwood). 2015;34(5):819-827. doi:10.1377/hlthaff.2014.1084 PubMedGoogle ScholarCrossref
16.
Elliott  MN, Lehrman  WG, Goldstein  E, Hambarsoomian  K, Beckett  MK, Giordano  LA.  Do hospitals rank differently on HCAHPS for different patient subgroups?   Med Care Res Rev. 2010;67(1):56-73. doi:10.1177/1077558709339066 PubMedGoogle ScholarCrossref
17.
Medicare Health Outcomes Survey. Centers for Medicare & Medicaid Services. Updated April 14, 2021. Accessed May 4, 2021. https://www.hosonline.org/
18.
Medicare Advantage and Prescription Drug Plan CAHPS Survey. Centers for Medicare & Medicaid Services. Updated March 29, 2021. Accessed May 4, 2021. https://www.ma-pdpcahps.org/
19.
Healthcare Effectiveness Data and Information Set RIF. Research Data Assistance Center. Accessed December 1, 2020. https://resdac.org/cms-data/files/hedis-rif
20.
The National Academies of Sciences, Engineering, and Medicine.  Accounting for Social Risk Factors in Medicare Payment: Criteria, Factors, and Methods. The National Academies Press; 2016.
21.
Samson  LW, Finegold  K, Ahmed  A, Jensen  M, Filice  CE, Joynt  KE.  Examining measures of income and poverty in Medicare administrative data.   Med Care. 2017;55(12):e158-e163. doi:10.1097/MLR.0000000000000606 PubMedGoogle ScholarCrossref
22.
Medicare 2019 Part C & D Star Ratings Technical Notes. Centers for Medicare & Medicaid Services. Updated March 21, 2019. Accessed May 4, 2021. https://www.cms.gov/Medicare/Prescription-Drug-Coverage/PrescriptionDrugCovGenIn/Downloads/2019-Technical-Notes.pdf
23.
Anderson  RE, Ayanian  JZ, Zaslavsky  AM, McWilliams  JM.  Quality of care and racial disparities in Medicare among potential ACOs.   J Gen Intern Med. 2014;29(9):1296-1304. doi:10.1007/s11606-014-2900-3 PubMedGoogle ScholarCrossref
24.
Lawson  EH, Ko  CY, Adams  JL, Chow  WB, Hall  BL.  Reliability of evaluating hospital quality by colorectal surgical site infection type.   Ann Surg. 2013;258(6):994-1000. doi:10.1097/SLA.0b013e3182929178 PubMedGoogle ScholarCrossref
25.
Abel  G, Saunders  CL, Mendonca  SC, Gildea  C, McPhail  S, Lyratzopoulos  G.  Variation and statistical reliability of publicly reported primary care diagnostic activity indicators for cancer: a cross-sectional ecological study of routine data.   BMJ Qual Saf. 2018;27(1):21-30. doi:10.1136/bmjqs-2017-006607 PubMedGoogle ScholarCrossref
26.
Zaslavsky  AM.  Statistical issues in reporting quality data: small samples and casemix variation.   Int J Qual Health Care. 2001;13(6):481-488. doi:10.1093/intqhc/13.6.481 PubMedGoogle ScholarCrossref
27.
Report to Congress: Medicare and the health care delivery system: redesigning the Medicare Advantage quality bonus program. MedPAC. June 2019. Accessed May 4, 2021. http://www.medpac.gov/docs/default-source/reports/jun19_ch8_medpac_reporttocongress_sec.pdf
28.
Alsan  M, Garrick  O, Graziani  G.  Does diversity matter for health? experimental evidence from Oakland.   Am Econ Rev. 2019;109(12):4071-4111. doi:10.1257/aer.20181446 Google ScholarCrossref
29.
Meyers  DJ, Rahman  M, Trivedi  AN.  Narrow primary care networks in Medicare Advantage.   J Gen Intern Med. Published online January 19, 2021. doi:10.1007/s11606-020-06534-2 PubMedGoogle Scholar
30.
Racial, ethnic, and gender disparities in health care in Medicare Advantage. Centers for Medicare & Medicaid Services Office of Minority Health in collaboration with the RAND Corporation. April 2018. Accessed May 4, 2021. https://www.cms.gov/About-CMS/Agency-Information/OMH/Downloads/2018-National-Level-Results-by-Race-Ethnicity-and-Gender.pdf
31.
Martino  SC, Weinick  RM, Kanouse  DE,  et al.  Reporting CAHPS and HEDIS data by race/ethnicity for Medicare beneficiaries.   Health Serv Res. 2013;48(2 pt 1):417-434. doi:10.1111/j.1475-6773.2012.01452.x PubMedGoogle ScholarCrossref
32.
Agniel  D, Martino  SC, Burkhart  Q,  et al.  Incentivizing excellent care to at-risk groups with a health equity summary score.   J Gen Intern Med. Published online November 11, 2019. doi:10.1007/s11606-019-05473-x PubMedGoogle Scholar
33.
Zaslavsky  AM, Ayanian  JZ, Zaborski  LB.  The validity of race and ethnicity in enrollment data for Medicare beneficiaries.   Health Serv Res. 2012;47(3 pt 2):1300-1321. doi:10.1111/j.1475-6773.2012.01411.x PubMedGoogle ScholarCrossref
34.
Jarrín  OF, Nyandege  AN, Grafova  IB, Dong  X, Lin  H.  Validity of race and ethnicity codes in Medicare administrative data compared with gold-standard self-reported race collected during routine home health care visits.   Med Care. 2020;58(1):e1-e8. doi:10.1097/MLR.0000000000001216 PubMedGoogle ScholarCrossref
35.
Filice  CE, Joynt  KE.  Examining race and ethnicity information in Medicare administrative data.   Med Care. 2017;55(12):e170-e176. doi:10.1097/MLR.0000000000000608 PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Views 4,360
    Original Investigation
    June 11, 2021

    Association of Medicare Advantage Star Ratings With Racial, Ethnic, and Socioeconomic Disparities in Quality of Care

    Author Affiliations
    • 1Department of Health Services, Policy, and Practice, Brown University School of Public Health, Providence, Rhode Island
    • 2Providence VA Medical Center, Providence, Rhode Island
    JAMA Health Forum. 2021;2(6):e210793. doi:10.1001/jamahealthforum.2021.0793
    Key Points

    Question  Do Medicare Advantage star ratings, which are generated using data from all enrollees in a plan, reflect the quality experience of racial/ethnic minorities and enrollees with low socioeconomic status (SES)?

    Findings  This cross-sectional study of 1 578 564 Medicare Advantage enrollees found that simulated star ratings for persons with lower SES and Black and Hispanic enrollees were substantially lower than ratings for those with higher SES and White enrollees in the same contract. There was little correlation between ratings for each group, and higher-performing plans, as measured by the star rating, had larger racial/ethnic and SES disparities in care.

    Meaning  Medicare Advantage star ratings, which are designed to reflect overall performance in a plan, are only modestly associated with quality for racial/ethnic minorities and enrollees of low SES in the same plan.

    Abstract

    Importance  Medicare Advantage (MA) plans, which disproportionately enroll racial/ethnic minorities and persons with socioeconomic disadvantage, receive bonus payments on the basis of overall performance on a 5-star rating scale. The association between plans’ overall quality and disparities in quality is not well understood.

    Objective  To examine the association between MA star ratings and disparities in care for racial/ethnic minorities and enrollees with lower income and educational attainment.

    Design, Setting, and Participants  This cross-sectional study included 1 578 564 MA enrollees from 454 contracts across the 2015 and 2016 calendar years. Data analyses were conducted between June 2019 and June 2020.

    Exposures  Self-reported race and ethnicity and low socioeconomic status (SES) (defined by low income or less than a high school education) vs high SES (neither low income nor low educational attainment).

    Main Outcomes and Measures  Performance on 22 measures of quality and satisfaction determined at the individual enrollee level, aggregated into simulated star ratings (scale, 2-5) stratified by SES and race/ethnicity.

    Results  A total of 1 578 564 enrollees were included in this analysis (55.8% female; mean [SD] age, 71.4 [11.3] years; 65.8% White; 12.3% Black; 14.6% Hispanic). Enrollees with low SES had simulated stratified star ratings 0.5 stars lower (95% CI, 0.4-0.6 stars) than individuals with high SES in the same contract. Black and Hispanic enrollees had simulated star ratings that were 0.3 stars (95% CI, 0.2-0.4 stars) and 0.1 stars (95% CI, −0.04 to 0.2 stars) lower than White enrollees within the same contracts. Black enrollees had a 0.4-star lower rating (95% CI, 0.1-0.7 stars) in 4.5- to 5-star contracts and a no statistical difference in 2.0- to 2.5-star–rated contracts (difference, 0.3 stars; 95% CI, −0.02 to 0.7 stars). Hispanic enrollees had a 0.6-star lower simulated rating (95% CI, 0.2-1.0 stars) in 4.5- to 5-star contracts and no statistical difference in 2- to 2.5-star contracts (difference, −0.01 stars; 95% CI, −0.5 to 0.4 stars). There was low correlation between simulated ratings for enrollees of low SES and high SES (difference, 0.2 stars; 95% CI, 0.03-0.4 stars) and between simulated ratings for White and Black enrollees (difference, 0.4 stars; 95% CI, 0.3-0.5 stars) and White and Hispanic enrollees (difference, 0.3 stars; 95% CI, 0.2-0.4 stars). As the proportion of Black and Hispanic enrollees increased in a contract, racial/ethnic disparities in ratings decreased.

    Conclusions and Relevance  In this cross-sectional study, simulated MA star ratings were only weakly correlated with those for enrollees of low SES in the same contract, and contracts with higher star ratings had larger disparities in quality. Measures of equity in MA plans’ quality of care may be needed.

    Introduction

    More than one-third of Medicare beneficiaries were enrolled in Medicare Advantage (MA) in 2019.1,2 In MA, private plans receive capitated payments to cover their enrollees’ health care needs. Medicare Advantage plans enroll higher proportions of racial/ethnic minorities and enrollees with lower income and education than the traditional Medicare program, and prior work has found substantial disparities in the quality of care in the MA program.3-14 These disparities in care have been found within plans (disparities in quality of care for enrollees in the same plans), and between plans (disparities driven by disproportionate enrollment of minorities in plans with worse quality).3,13,14

    Since 2008, the US Centers for Medicare & Medicaid Services (CMS) has used a 5-star rating system to measure the performance of MA contracts and allocates $6 billion in annual bonus payments on the basis of these star ratings. Then CMS calculates these ratings and assigns bonus payments for all enrollees in a contract without stratifying results by race, ethnicity, or socioeconomic status (SES). While the CMS Office of Minority Health reports some individual measures by race/ethnicity dual status, as well as by disability, this stratification does not contribute to plan payment decisions. If aggregate contract star ratings hide clinically important differences in quality between advantaged and disadvantaged plan members, then quality measures that directly assess equity may be needed.15,16 This study addressed 2 main points: first, if there is an association between an MA contract’s overall star rating and what the star rating would have been if calculated for that contract’s enrollees who are racial/ethnic minorities or who have lower SES; second, if contracts with higher star ratings have lower disparities in care.

    Methods
    Data Sources

    We compiled individual-level data on 22 measures included in CMS’s current star-rating calculation using 4 primary sources from the 2015 and 2016 calendar years. First, we used the Medicare Health Outcomes Survey (HOS), which randomly samples 1200 enrollees from each MA contract.17 Five variables from HOS are used in the star rating (eTable 1 in the Supplement). We focused on measures included in MA star ratings and did not include measures that evaluate Part D benefits, as not all plans include Part D benefits. Second, we used the MA Consumer Assessment of Healthcare Providers and Systems (CAHPS), an annual cross-sectional survey of 600 enrollees from each MA contract.18 Third, we used 10 person-level measures from the MA Healthcare Effectiveness Data and Information Set files.19 We linked beneficiaries to the Master Beneficiary Summary File (MBSF) to identify their MA contract and to calculate our measure of disenrollment. We considered an enrollee as disenrolled from the plan if in the following year they were enrolled in a different MA plan or in traditional Medicare, excluding those who moved counties between years and those in plans that exited the MA market.

    This study was approved by the institutional review board at Brown University and received a waiver of informed consent owing to use of deidentified data. Analyses took place between June 2019 and June 2020, and Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guidelines were followed.

    Classifying SES and Race/Ethnicity

    Both the CAHPS and HOS include questions on SES and self-reported race/ethnicity. We selected low income and less than high school education as measures of low SES; both have been identified by the National Academy of Medicine as key social determinants of health.20 We classified an individual as having low income if they were dually eligible (including both full and partial) with Medicaid or were eligible for the Part D low-income subsidy as specified in either the HOS, CAHPS, or MBSF.21 We identified individuals with less than a high school education based on self-reported questions in both surveys. We stratified the results by enrollees who had both low income and low education (both low SES) compared with those who had either low income or low education (either low SES) and those who had neither low income nor low education (high SES).

    Both the CAHPS and HOS include self-reported race/ethnicity data. To align with CMS’s methodologies, we classified enrollees who self-reported their ethnicity as Hispanic as Hispanic, regardless of what race they selected. We classified an enrollee as non-Hispanic Black (hereafter Black) if they reported their race as Black in either the HOS or CAHPS and did not report Hispanic ethnicity. For the race/ethnicity comparisons, the comparison group comprised those who self-identified as non-Hispanic White. We report the agreement between the self-reported and MBSF designations in eTable 3 in the Supplement.

    Study Population

    We included all beneficiaries who responded to the HOS and CAHPS surveys in 2015 and 2016 in our analysis and combined the years to provide a larger sample. We excluded enrollees who were enrolled in employee-sponsored MA contracts and in Medicare-Medicaid fully integrated plans. To ensure the stability of our analysis, we required that every contract have at least 50 sampled enrollees from each SES and race/ethnicity group. In sensitivity analyses, we also tested requiring at least 100 enrollees from each group to test the stability of our estimates. Response rates did not vary substantially by survey or study population (eTable 5 in the Supplement).

    Calculation of Simulated Star Ratings

    To determine a star rating, CMS calculates each contract on the basis of 30 to 35 measures of satisfaction and quality outcomes.22 Then CMS assigns ratings at the contract level. Each contract may contain with it any number of plans that may vary in their benefits and cost sharing. We performed our analysis at the contract level, consistent with CMS’s rating methods (eMethods in the Supplement). We replicated CMS’s approach with 22, or 70%, of all the measures included in star-rating calculation based on 2016 cut points. When CMS calculates star ratings, the survey data and the cut points to assign different measures vary from year to year. In this study, we use 2016 cut points to estimate 2016 ratings; however, because these survey data are from across several years, our simulated ratings are not identical to those of CMS. We tested alterative methods for calculating the ratings using 2015 and 2017 cut points, and by averaging 2015, 2016, and 2017 cut points, but the results did not vary substantially. Our simulated overall star rating was highly correlated with the official 2015 and 2016 star ratings (both correlations = 0.9). After calculating a simulated overall star rating, we calculated stratified star ratings using only enrollees in each of the 5 stratified groups (high SES, low SES, White, Black, and Hispanic).

    Statistical Analysis

    We calculated the difference between each contract’s simulated star rating for high-SES and low-SES groups, and between each contract’s White and Black or Hispanic enrollees. We compared differences in contract-level ratings for each of these groups using analysis of variance, χ2 statistical tests, and 95% CIs. Given the complexity of the different data sources that are included in this analysis, we calculated all SEs from bootstrapping with 10 000 replications. We calculated the sample correlation between measures using Pearson correlation coefficients.

    We also decomposed any disparity in ratings into within-contract and between-contract differences following methods previously described in the literature.3,23 The within-contract difference was calculated as the contract-level difference in the rating of each group, averaged across all contracts. We calculated the between-contract difference by taking the mean of the contract-level differences weighted by the number of Black or Hispanic enrollees and enrollees of low SES in each contract. We then subtracted this weighted difference from the total difference to calculate the between disparity.23 The within-contract disparity can be interpreted as the disparity in outcomes attributable to differences in outcomes among persons enrolled within the same contract, while the between-contract disparity can be interpreted as the disparity in outcomes attributable to persons with low SES or racial/ethnic minorities being disproportionately enrolled in contracts with lower star ratings.

    We intentionally did not include any adjustments from multivariable regression analysis in this study for 2 reasons. First, most measures included in CMS star-rating calculation do not use adjustment, and we wanted to simulate CMS’s methods. Second, while adjustment can help compare performance on a more even playing field, it may also obscure absolute differences in performance between groups. We elect to stratify the simulated results to better highlight where differences exist.

    In sensitivity analyses, we compared the sample sizes included in each measure; compared results when excluding Puerto Rico, because the MA market and disparities within plans may function differently outside the continental US9; and compared absolute differences between simulated ratings and individual measures within contract. We also calculated the interunit reliabilities for each measure across contracts and strata to assess to what extent disparities between contracts may be related to chance (eTable 8 in the Supplement).24-26 All analyses were conducted in Stata, version 16 (StataCorp), and a 2-sided α = .05 was considered statistically significant.

    Results

    Table 1 summarizes the demographic and contract characteristics of the 1 578 564 MA enrollees (55.8% female; mean [SD] age, 71.4 [11.3] years) included in the analysis, stratified by SES and race/ethnicity. Of the enrollees in the sample, 7.0% were of low SES, and 12.3% and 14.6% of the sample were Black and Hispanic, respectively.

    Table 2 summarizes the mean simulated stratified star ratings at the contract level. Enrollees of low SES had a 0.5-star lower simulated rating than enrollees of high SES in the same contracts (95% CI, 0.4-0.6 stars). Black enrollees had a 0.3-star lower simulated rating on average than White enrollees in the same contract (95% CI, 0.2-0.4 stars). For enrollees of low SES and Black enrollees, more of the disparity in star ratings could be attributed to within-contract differences (66% and 54%, respectively). Conversely, for Hispanic enrollees, the largest disparity was attributable to between-contract differences (85%). Within-contract differences for individual measures are presented in eTable 6 in the Supplement.

    Figure 1 shows the association between contract-level simulated star ratings for enrollees of high SES and low SES, as well as ratings for White and racial/ethnic minority enrollees. In 71% of contracts, enrollees of low SES had a lower stratified rating than those with high SES. The stratified star rating for Black enrollees was lower than the rating for White enrollees in 57% of contracts, and the rating for Hispanic enrollees was lower than that of White enrollees in 49% of contracts. The correlation coefficient between the star rating for enrollees of both low and high SES was 0.2 (95% CI, 0.03-0.4). The correlation coefficient between the rating for Black enrollees and the rating for White enrollees was 0.4 (95% CI, 0.3-0.5), and between Hispanic enrollees and White enrollees was 0.3 (95% CI, 0.2-0.4).

    Figure 2 compares the disparity in simulated star ratings between groups by decile of each group’s concentration. In contracts in the lowest decile of low-SES enrollment, enrollees of low SES had a 0.9-star lower rating than enrollees of high SES (95% CI, 0.4-1.3 stars; P < .001) compared with contracts in the highest decile of low-SES enrollment, where enrollees of low SES had a 0.6-star higher rating than enrollees of high SES (95% CI, 0.2-1.0 stars; P < .001). In contracts in the lowest decile of Black enrollment, Black enrollees had a 0.9-star lower rating than White enrollees (95% CI, 0.5-1.2 stars; P < .001) compared with contracts in the highest decile of Black enrollment, where Black enrollees had a 0.3-star higher rating than White enrollees (95% CI, −0.02 to 0.6 stars; P < .001). In contracts in the lowest decile of Hispanic enrollment, Hispanic enrollees had a 0.6-star lower rating than White enrollees (95% CI, 0.2-1.0 stars; P < .001) compared with contracts in the highest decile of Hispanic enrollment, where Hispanic enrollees had a 0.8-star higher rating than White enrollees (95% CI, 0.5-1.2 stars; P < .001). The stratified ratings for each group across deciles is presented in eFigure 1 in the Supplement. As the decile of low SES, Black, and Hispanic increases, so does the stratified rating for low SES, Black, and Hispanic enrollees. The cut points for each decile are included in eTable 2 in the Supplement.

    Figure 3 plots the disparity in simulated star ratings by the official star rating assigned to each contract in 2015. In 4.5- to 5-star contracts, enrollees of low SES had a 1.0-star lower rating (95% CI, 0.4-1.6 stars) than enrollees of high SES. The disparity was not statistically significant in 2- to 2.5-star contracts (difference, 0.1; 95% CI, −0.4 to 0.7). Black enrollees had a 0.4-star lower rating (95% CI, 0.1-0.7 stars) than White enrollees within 4.5- to 5-star contracts and no statistical difference compared with White enrollees within 2.0- to 2.5-star contracts (difference, 0.3; 95% CI, −0.02 to 0.7). Hispanic enrollees had a 0.6-star lower rating (95% CI, 0.2-1.0 stars) in 4.5- to 5-star contracts and no statistical difference in 2- to 2.5-star contracts (difference, −0.01; 95% CI, −0.5 to 0.4). eFigure 2 in the Supplement presents the stratified ratings by official star rating, and the sensitivity analyses yielded similar results.

    Discussion

    This cross-sectional study had 4 key findings. First, we observed only a modest correlation of simulated star ratings when calculated for enrollees of low SES and high SES, and between racial/ethnic minority enrollees and White enrollees in the same contract. Second, contracts with higher star ratings had larger racial/ethnic disparities than did those with lower star ratings. Third, the contracts with lower concentrations of individuals of low SES and Black or Hispanic individuals had larger disparities and worse quality for these individuals. Fourth, we identified both within-plan and between-plan disparities in the quality of care in the MA program, as measured by the star ratings.

    We build on past work3,10 detailing disparities in the MA program in several key ways. First, we found that the disparities are evident not just in some selected outcomes,3,10 but across aggregate plan quality and in a composite metric (the star rating) that determines the distribution of $6 billion in annual bonus payments to MA plans.27 Second, to our knowledge, this is the first study to demonstrate that MA plans with higher-measured quality have larger magnitudes of disparity in quality within their enrolled populations. Third, we found a low correlation between a plan’s rating for its enrollees of low SES and Black and Hispanic enrollees, and its enrollees of high SES and White enrollees, although this may be largely due to the low reliability of the simulated scores.

    There are many factors that could explain disparities in quality, including access to care, plan cultural competence, access to high-quality or racially concordant health care professionals,28,29 and other facets of structural racism. The CMS star ratings currently reward contracts that have greater disparities in quality for enrollees of low SES and Black or Hispanic enrollees. Without stratification, contracts may not be aware of their performance for racial/ethnic minorities and for persons with social risk factors. In an effort to avoid penalizing contracts that serve greater concentrations of individuals of low SES, in 2017 CMS implemented a Categorical Adjustment Index to adjust the calculation of star ratings based on the proportion of dual eligibility in a contract.14 This adjustment may make quality bonus payments more equitable across contracts, but it does not replace the benefit of stratification, which may help to better illustrate where disparities exist

    This study is among the first, to our knowledge, that finds variation in disparities by some contract characteristics. We find that contracts with high concentrations of individuals of low SES and Black or Hispanic individuals actually perform better for those populations in stratified star ratings than they do for individuals of high SES and White individuals. It is possible that contracts that treat large numbers of these populations offer additional supplemental benefits or tailored interventions to better address their needs. They may also contract with insurance provider networks that perform better for these populations. Future work should seek to understand what other contract and plan-level factors may contribute to mitigate disparities in plans and the mechanisms that explain why disparities are larger in plans with higher star ratings and fewer racial/ethnic minority enrollees.

    While we did find that certain contracts have larger disparities than others, these contracts may still be preferable to marginalized enrollees if they deliver better quality outcomes. However, we found that for contracts with higher proportions of Black or Hispanic enrollees and enrollees of low SES, the disparity both decreases and the absolute performance is better for Black or Hispanic enrollees and enrollees with low income (eFigure 1 in the Supplement). When comparing overall star ratings, higher-rated contracts do perform better for all enrollees; however, the disparities in care are larger, so the potential is not evident for all enrollees (eFigure 2 in the Supplement).

    Several policy solutions could improve the measurement of quality in MA. First, CMS could publicly report on stratified star ratings, as we have herein. The CMS Office of Minority Health reports on disparities in measures across the entire MA program and provides stratified performance on specific measures at the contract level, but it does not publicly generate overall stratified star ratings or report on all outcomes on which contracts are evaluated.30 By publishing stratified star ratings, plans may have an incentive to improve the outcomes for their enrollees, and enrollees may also be able to make more informed decisions about which contracts will best serve them.31 While a plan may be able to take specific action on stratified individual measures, it may be separately beneficial to stratify aggregated star ratings. A stratified star rating may be more easily used by an enrollee seeking a plan that would most benefit them and may also do a better job of capturing the overall culture of health that a plan is producing than individual measures.

    An alternative solution would be to formally include a measure of health equity in the calculation of star ratings,32 providing an increased overall rating to contracts that have lower disparities in care or that are able to reduce their disparities over time. This metric could be implemented in a similar fashion to current adjustments for plans that improve their overall quality, or included as a separate stand-alone measure incorporated into star ratings. An equity measure may create an incentive for a contract to reduce disparities without creating an incentive for plans to avoid enrollees who have social risk factors. The Medicare Payment Advisory Commission has suggested eliminating the 5-star rating system altogether, instead replacing it with a more parsimonious set of population-based measures.27 The present results highlight that any future form of MA quality measurement will likely require more focused attention on disparities.

    Limitations

    This study has several limitations. First, we were unable to calculate ratings using the exact methods as CMS owing to restrictions in the years of available data and several measures that are not available to researchers. Our work here uses the CMS star system as a familiar method for summarizing quality information across measures and domains, but we do not present it as the only solution to the problem of measuring and incentivizing equitable care. Second, as a result of our exclusion criteria requiring at least 50 enrollees of each type to be included in an analysis, not all MA contracts were included in the final analysis (eTable 4 in the Supplement). Third, even with a requirement of including at least 50 enrollees in each contract for analysis, the sample sizes for some comparisons may be small (eTable 7 in the Supplement), which could lead to instability in our estimates. In our reliability analysis we found that some measures for some contracts may not have been reliable; however, when excluding these measures from our simulated ratings, the differences were minor. Fourth, Black and Hispanic enrollees had lower response rates to the CAHPS and HOS than White enrollees, though it is unclear if additional responses would widen or narrow the disparities we measured. Fifth, we intentionally combined Hispanic ethnicity and race into 1 variable to align with CMS’s current methodologies; however, these identities are multidimensional, and this approach may not have captured the experience of all beneficiaries. This study (eTable 3 in the Supplement) and others33,34 have found that there is often disagreement between self-report and CMS’s race/ethnicity measures, and they highlight the importance of using self-reported race/ethnicity data. Despite these limitations, this is the first effort that we are aware of to calculate star ratings stratified by self-reported enrollee type and to use these star ratings to demonstrate within-contract and between-contract differences.21,35

    Conclusions

    Results of this cross-sectional study found that simulated star ratings for White enrollees and those with higher SES are only modestly associated with star ratings for minority enrollees and those with lower SES in the same contract. Contracts with higher Medicare star ratings have larger racial, ethnic, and socioeconomic disparities in quality. These findings indicate that the MA star ratings may need to be modified to explicitly consider and reward equity in care.

    Back to top
    Article Information

    Accepted for Publication: April 6, 2021.

    Published: June 11, 2021. doi:10.1001/jamahealthforum.2021.0793

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Meyers DJ et al. JAMA Health Forum.

    Corresponding Author: David J. Meyers, PhD, MPH, Department of Health Services, Policy, and Practice, Brown University School of Public Health, 121 S Main St, Providence, RI 02912 (david_meyers@brown.edu).

    Author Contributions: Dr Meyers had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: All authors.

    Acquisition, analysis, or interpretation of data: Meyers, Rahman, Mor, Trivedi.

    Drafting of the manuscript: Meyers, Rahman.

    Critical revision of the manuscript for important intellectual content: Meyers, Mor, Wilson, Trivedi.

    Statistical analysis: Meyers, Rahman.

    Obtained funding: Meyers.

    Administrative, technical, or material support: Meyers, Mor.

    Supervision: Rahman, Mor, Wilson, Trivedi.

    Conflict of Interest Disclosures: Dr Meyers reported receiving grants from the Agency for Healthcare Research and Quality during the conduct of the study and personal fees from NORC outside the submitted work. Dr Rahman reported receiving grants from the National Institute on Aging during the conduct of the study. Prof Mor reported receiving grants from the National Institute on Aging during the conduct of the study, receiving personal fees as the chair of the scientific advisory committee at naviHealth Inc, and is the former chair of the independent quality committee at HCR ManorCare and former director of PointRight Inc, where he holds less than 1% equity. Dr Wilson reported receiving grants from the National Institute of Mental Health during the conduct of the study. Dr Trivedi reported receiving grants from the National Institute on Aging during the conduct of the study and grants from the Agency for Healthcare Research and Quality outside the submitted work.

    Funding/Support: This work was supported by a grant from the Agency for Healthcare Research and Quality (R36HS02705101).

    Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Disclaimer: The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the US Department of Veterans Affairs or the US government.

    Additional Information: Data used in this study were granted under a data-use agreement with the US Centers for Medicare & Medicaid Services and cannot be provided without its consent. Code used in the calculation of these results are available from the corresponding author on request.

    References
    1.
    Neuman  P, Jacobson  GA.  Medicare Advantage checkup.   N Engl J Med. 2018;379(22):2163-2172. doi:10.1056/NEJMhpr1804089 PubMedGoogle ScholarCrossref
    2.
    Meyers  DJ, Johnston  KJ.  The Growing Importance of Medicare Advantage in Health Policy and Health Services Research.   JAMA Health Forum. 2021;2(3):e210235. doi:10.1001/jamahealthforum.2021.0235Google Scholar
    3.
    Trivedi  AN, Zaslavsky  AM, Schneider  EC, Ayanian  JZ.  Relationship between quality of care and racial disparities in Medicare health plans.   JAMA. 2006;296(16):1998-2004. doi:10.1001/jama.296.16.1998 PubMedGoogle ScholarCrossref
    4.
    Weech-Maldonado  R, Elliott  MN, Adams  JL,  et al.  Do racial/ethnic disparities in quality and patient experience within Medicare plans generalize across measures and racial/ethnic groups?   Health Serv Res. 2015;50(6):1829-1849. doi:10.1111/1475-6773.12297 PubMedGoogle ScholarCrossref
    5.
    Li  Y, Cen  X, Cai  X, Thirukumaran  CP, Zhou  J, Glance  LG.  Medicare Advantage associated with more racial disparity than traditional Medicare for hospital readmissions.   Health Aff (Millwood). 2017;36(7):1328-1335. doi:10.1377/hlthaff.2016.1344 PubMedGoogle ScholarCrossref
    6.
    Martino  SC, Elliott  MN, Hambarsoomian  K,  et al.  Racial/ethnic disparities in Medicare beneficiaries’ care coordination experiences.   Med Care. 2016;54(8):765-771. doi:10.1097/MLR.0000000000000556 PubMedGoogle ScholarCrossref
    7.
    Ayanian  JZ, Landon  BE, Zaslavsky  AM, Newhouse  JP.  Racial and ethnic differences in use of mammography between Medicare Advantage and traditional Medicare.   J Natl Cancer Inst. 2013;105(24):1891-1896. doi:10.1093/jnci/djt333 PubMedGoogle ScholarCrossref
    8.
    Elliott  MN, Haviland  AM, Orr  N, Hambarsoomian  K, Cleary  PD.  How do the experiences of Medicare beneficiary subgroups differ between managed care and original Medicare?   Health Serv Res. 2011;46(4):1039-1058. doi:10.1111/j.1475-6773.2011.01245.x PubMedGoogle ScholarCrossref
    9.
    Rivera-Hernandez  M, Leyva  B, Keohane  LM, Trivedi  AN.  Quality of care for White and Hispanic Medicare Advantage enrollees in the United States and Puerto Rico.   JAMA Intern Med. 2016;176(6):787-794. doi:10.1001/jamainternmed.2016.0267 PubMedGoogle ScholarCrossref
    10.
    Ayanian  JZ, Landon  BE, Newhouse  JP, Zaslavsky  AM.  Racial and ethnic disparities among enrollees in Medicare Advantage plans.   N Engl J Med. 2014;371(24):2288-2297. doi:10.1056/NEJMsa1407273 PubMedGoogle ScholarCrossref
    11.
    Chou  AF, Brown  AF, Jensen  RE, Shih  S, Pawlson  G, Scholle  SH.  Gender and racial disparities in the management of diabetes mellitus among Medicare patients.   Womens Health Issues. 2007;17(3):150-161. doi:10.1016/j.whi.2007.03.003 PubMedGoogle ScholarCrossref
    12.
    Abraham  JM, Marmor  S, Knutson  D, Zeglin  J, Virnig  B.  Variation in diabetes care quality among Medicare Advantage plans: understanding the role of case mix.   Am J Med Qual. 2012;27(5):377-382. doi:10.1177/1062860611428529 PubMedGoogle ScholarCrossref
    13.
    Breslau  J, Elliott  MN, Haviland  AM,  et al.  Racial and ethnic differences in the attainment of behavioral health quality measures in Medicare Advantage plans.   Health Aff (Millwood). 2018;37(10):1685-1692. doi:10.1377/hlthaff.2018.0655 PubMedGoogle ScholarCrossref
    14.
    Sorbero  ME, Paddock  SM, Damberg  CL,  et al.  Adjusting Medicare Advantage star ratings for socioeconomic status and disability.   Am J Manag Care. 2018;24(9):e285-e291.PubMedGoogle Scholar
    15.
    Konetzka  RT, Grabowski  DC, Perraillon  MC, Werner  RM.  Nursing home 5-star rating system exacerbates disparities in quality, by payer source.   Health Aff (Millwood). 2015;34(5):819-827. doi:10.1377/hlthaff.2014.1084 PubMedGoogle ScholarCrossref
    16.
    Elliott  MN, Lehrman  WG, Goldstein  E, Hambarsoomian  K, Beckett  MK, Giordano  LA.  Do hospitals rank differently on HCAHPS for different patient subgroups?   Med Care Res Rev. 2010;67(1):56-73. doi:10.1177/1077558709339066 PubMedGoogle ScholarCrossref
    17.
    Medicare Health Outcomes Survey. Centers for Medicare & Medicaid Services. Updated April 14, 2021. Accessed May 4, 2021. https://www.hosonline.org/
    18.
    Medicare Advantage and Prescription Drug Plan CAHPS Survey. Centers for Medicare & Medicaid Services. Updated March 29, 2021. Accessed May 4, 2021. https://www.ma-pdpcahps.org/
    19.
    Healthcare Effectiveness Data and Information Set RIF. Research Data Assistance Center. Accessed December 1, 2020. https://resdac.org/cms-data/files/hedis-rif
    20.
    The National Academies of Sciences, Engineering, and Medicine.  Accounting for Social Risk Factors in Medicare Payment: Criteria, Factors, and Methods. The National Academies Press; 2016.
    21.
    Samson  LW, Finegold  K, Ahmed  A, Jensen  M, Filice  CE, Joynt  KE.  Examining measures of income and poverty in Medicare administrative data.   Med Care. 2017;55(12):e158-e163. doi:10.1097/MLR.0000000000000606 PubMedGoogle ScholarCrossref
    22.
    Medicare 2019 Part C & D Star Ratings Technical Notes. Centers for Medicare & Medicaid Services. Updated March 21, 2019. Accessed May 4, 2021. https://www.cms.gov/Medicare/Prescription-Drug-Coverage/PrescriptionDrugCovGenIn/Downloads/2019-Technical-Notes.pdf
    23.
    Anderson  RE, Ayanian  JZ, Zaslavsky  AM, McWilliams  JM.  Quality of care and racial disparities in Medicare among potential ACOs.   J Gen Intern Med. 2014;29(9):1296-1304. doi:10.1007/s11606-014-2900-3 PubMedGoogle ScholarCrossref
    24.
    Lawson  EH, Ko  CY, Adams  JL, Chow  WB, Hall  BL.  Reliability of evaluating hospital quality by colorectal surgical site infection type.   Ann Surg. 2013;258(6):994-1000. doi:10.1097/SLA.0b013e3182929178 PubMedGoogle ScholarCrossref
    25.
    Abel  G, Saunders  CL, Mendonca  SC, Gildea  C, McPhail  S, Lyratzopoulos  G.  Variation and statistical reliability of publicly reported primary care diagnostic activity indicators for cancer: a cross-sectional ecological study of routine data.   BMJ Qual Saf. 2018;27(1):21-30. doi:10.1136/bmjqs-2017-006607 PubMedGoogle ScholarCrossref
    26.
    Zaslavsky  AM.  Statistical issues in reporting quality data: small samples and casemix variation.   Int J Qual Health Care. 2001;13(6):481-488. doi:10.1093/intqhc/13.6.481 PubMedGoogle ScholarCrossref
    27.
    Report to Congress: Medicare and the health care delivery system: redesigning the Medicare Advantage quality bonus program. MedPAC. June 2019. Accessed May 4, 2021. http://www.medpac.gov/docs/default-source/reports/jun19_ch8_medpac_reporttocongress_sec.pdf
    28.
    Alsan  M, Garrick  O, Graziani  G.  Does diversity matter for health? experimental evidence from Oakland.   Am Econ Rev. 2019;109(12):4071-4111. doi:10.1257/aer.20181446 Google ScholarCrossref
    29.
    Meyers  DJ, Rahman  M, Trivedi  AN.  Narrow primary care networks in Medicare Advantage.   J Gen Intern Med. Published online January 19, 2021. doi:10.1007/s11606-020-06534-2 PubMedGoogle Scholar
    30.
    Racial, ethnic, and gender disparities in health care in Medicare Advantage. Centers for Medicare & Medicaid Services Office of Minority Health in collaboration with the RAND Corporation. April 2018. Accessed May 4, 2021. https://www.cms.gov/About-CMS/Agency-Information/OMH/Downloads/2018-National-Level-Results-by-Race-Ethnicity-and-Gender.pdf
    31.
    Martino  SC, Weinick  RM, Kanouse  DE,  et al.  Reporting CAHPS and HEDIS data by race/ethnicity for Medicare beneficiaries.   Health Serv Res. 2013;48(2 pt 1):417-434. doi:10.1111/j.1475-6773.2012.01452.x PubMedGoogle ScholarCrossref
    32.
    Agniel  D, Martino  SC, Burkhart  Q,  et al.  Incentivizing excellent care to at-risk groups with a health equity summary score.   J Gen Intern Med. Published online November 11, 2019. doi:10.1007/s11606-019-05473-x PubMedGoogle Scholar
    33.
    Zaslavsky  AM, Ayanian  JZ, Zaborski  LB.  The validity of race and ethnicity in enrollment data for Medicare beneficiaries.   Health Serv Res. 2012;47(3 pt 2):1300-1321. doi:10.1111/j.1475-6773.2012.01411.x PubMedGoogle ScholarCrossref
    34.
    Jarrín  OF, Nyandege  AN, Grafova  IB, Dong  X, Lin  H.  Validity of race and ethnicity codes in Medicare administrative data compared with gold-standard self-reported race collected during routine home health care visits.   Med Care. 2020;58(1):e1-e8. doi:10.1097/MLR.0000000000001216 PubMedGoogle ScholarCrossref
    35.
    Filice  CE, Joynt  KE.  Examining race and ethnicity information in Medicare administrative data.   Med Care. 2017;55(12):e170-e176. doi:10.1097/MLR.0000000000000608 PubMedGoogle ScholarCrossref
    ×