Assessment of Disparities Associated With a Crisis Standards of Care Resource Allocation Algorithm for Patients in 2 US Hospitals During the COVID-19 Pandemic

Key Points

Question  Is there an association of race and/or ethnicity with priority scores based on both short-term and longer-term estimated mortality used for resource allocation under crisis standards of care?

Findings  In this retrospective cohort study of 1127 patients with 5613 patient-days in 2 US hospitals, there was no significant association of race or ethnicity with priority score.

Meaning  In this study, the use of a crisis standards of care resource allocation policy based on both short-term and longer-term estimated mortality did not appear to discriminate against hospitalized patients based on self-identified race or ethnicity.

Importance  Significant concern has been raised that crisis standards of care policies aimed at guiding resource allocation may be biased against people based on race/ethnicity.

Objective  To evaluate whether unanticipated disparities by race or ethnicity arise from a single institution’s resource allocation policy.

Design, Setting, and Participants  This cohort study included adults (aged ≥18 years) who were cared for on a coronavirus disease 2019 (COVID-19) ward or in a monitored unit requiring invasive or noninvasive ventilation or high-flow nasal cannula between May 26 and July 14, 2020, at 2 academic hospitals in Miami, Florida.

Exposures  Race (ie, White, Black, Asian, multiracial) and ethnicity (ie, non-Hispanic, Hispanic).

Main Outcomes and Measures  The primary outcome was based on a resource allocation priority score (range, 1-8, with 1 indicating highest and 8 indicating lowest priority) that was assigned daily based on both estimated short-term (using Sequential Organ Failure Assessment score) and longer-term (using comorbidities) mortality. There were 2 coprimary outcomes: maximum and minimum score for each patient over all eligible patient-days. Standard summary statistics were used to describe the cohort, and multivariable Poisson regression was used to identify associations of race and ethnicity with each outcome.

Results  The cohort consisted of 5613 patient-days of data from 1127 patients (median [interquartile range {IQR}] age, 62.7 [51.7-73.7]; 607 [53.9%] men). Of these, 711 (63.1%) were White patients, 323 (28.7%) were Black patients, 8 (0.7%) were Asian patients, and 31 (2.8%) were multiracial patients; 480 (42.6%) were non-Hispanic patients, and 611 (54.2%) were Hispanic patients. The median (IQR) maximum priority score for the cohort was 3 (1-4); the median (IQR) minimum score was 2 (1-3). After adjustment, there was no association of race with maximum priority score using White patients as the reference group (Black patients: incidence rate ratio [IRR], 1.00; 95% CI, 0.89-1.12; Asian patients: IRR, 0.95; 95% CI. 0.62-1.45; multiracial patients: IRR, 0.93; 95% CI, 0.72-1.19) or of ethnicity using non-Hispanic patients as the reference group (Hispanic patients: IRR, 0.98; 95% CI, 0.88-1.10); similarly, no association was found with minimum score for race, again with White patients as the reference group (Black patients: IRR, 1.01; 95% CI, 0.90-1.14; Asian patients: IRR, 0.96; 95% CI, 0.62-1.49; multiracial patients: IRR, 0.81; 95% CI, 0.61-1.07) or ethnicity, again with non-Hispanic patients as the reference group (Hispanic patients: IRR, 1.00; 95% CI, 0.89-1.13).

Conclusions and Relevance  In this cohort study of adult patients admitted to a COVID-19 unit at 2 US hospitals, there was no association of race or ethnicity with the priority score underpinning the resource allocation policy. Despite this finding, any policy to guide altered standards of care during a crisis should be monitored to ensure equitable distribution of resources.

Crisis standards of care (CSC) are necessary to allow for equitable and transparent allocation of limited resources during times of excess demand.^1,2 The coronavirus disease 2019 (COVID-19) pandemic has forced health care systems to confront the very real possibility that need for certain lifesaving resources (eg, intensive care unit [ICU] beds, ventilators, dialysis machines) may exceed supply. In response, regional governments³ and individual health care institutions⁴ revamped and, in some instances, de novo created CSC policies to aid in fair resource deployment.

While health care workers and lay people largely agree that triage following the default system of treating individuals on a first-come, first-served basis is not desirable,^5,6 there remains significant disagreement about how, exactly, scarce resource allocation should occur. Clinicians tend to favor policies aimed at prioritizing those who will likely both survive the current illness (ie, short-term prognosis) and live longer following recovery (ie, longer-term prognosis).⁵ Conversely, the general public favors aiming to save the most lives⁶ while also considering acute illness prognosis (either prioritizing those most likely to die without⁶ or survive with⁵ treatment) without a focus on longer-term prognosis. Most regional and institutional CSC policies incorporate some measure of estimated short-term survival (eg, based on Sequential Organ Failure Assessment [SOFA] scores⁷), and many, although not all, also include an assessment of likely longer-term prognosis (eg, based on comorbidities).^3,4

Significant concern has been raised that CSC policies—especially those that consider longer-term prognosis in triage scoring—may systematically deprioritize patients from underrepresented minority groups given the higher incidence of comorbidities among these populations resulting from systemic racism.^3,8,9 In fact, compared with White lay people, Black individuals were significantly more likely to prefer a triage algorithm based on the principle of first come, first served and less likely to prefer one aimed at saving the most life-years,⁵ which may be a reflection of this very real concern.

In this study, we sought to evaluate whether our institution’s CSC policy, which is based on both short-term and longer-term prognosis, would result in unintended deprioritization of patients from minority groups during COVID-19. Given that our algorithm groups short-term prognosis into broader groups and assigns longer-term prognosis scores based on the presence of 1 or more comorbidities, we hypothesized that race- and ethnicity-related differences would be minimized and no unintended disparities would result.

Data were collected as part of a quality improvement (QI) project aimed at evaluating the feasibility of implementing our newly created CSC policy, which depended on calculating daily priority scores for all patients at risk of mechanical ventilator triage due to surges in COVID-19 infection. We then conducted a retrospective cohort study of this data set. Data were collected daily from May 16 through July 14, 2020 from a midsize tertiary care hospital (May 26 through July 13, excluding May 31, June 20, and July 11) and a large quaternary care public hospital (June 30 through July 14, excluding July 6) at which University of Miami faculty attend.

Institutional review board approval was obtained from the University of Miami with a waiver of informed consent due to minimal risk to participants. The reporting of this work is consistent with the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline.¹⁰

A team of 2 medical ethicists (K.W.G. and J.P.B) and 3 pulmonary and critical care physicians (G.E.H., D.H.K., and H.B.G.) met over videoconferencing in March and April 2020 to refine a CSC policy that had been created (but not finalized) in preparation for Ebola virus disease in 2014. The portion of our CSC policy aimed at resource allocation was designed to mirror those publicly available across multiple states and to align with guidance from experts.^3,4 We had 3 primary goals in creating our policy: (1) to be fair and equitable; (2) to be actionable; and (3) to allocate scarce resources to those with both the greatest chance of surviving COVID-19 infection and living the longest. To this end, we created a primary allocation schema based on priority scores (1, indicating highest priority, through 8, indicating lowest priority) that were further consolidated into priority groups (1, indicating highest priority, through 3, indicating lowest priority) (Figure 1). Priority scores were a sum of points based on the likelihood of short-term mortality (based on daily SOFA score and categorized as 1-4 points, with 1 indicating a SOFA score of <6; 2, SOFA score 6-8; 3, SOFA score 9-11; 4, SOFA score ≥12) and longer-term mortality (based on comorbidities documented in the medical record, categorized as 0, 2, or 4 points). Points associated with comorbidities were assigned based on the likelihood of reduced 1-year (4 points) or 5-year (2 points) survival (eTable 1 in the Supplement). Patients only received 1 allotment of comorbidity points based the highest point value appropriate without a sum of scores from multiple comorbidities (ie, someone with 2 comorbidities with reduced 5-year survival and 3 comorbidities with reduced 1-year survival received 4 points for having at least 1 comorbidity that reduced 1-year survival). If needed, resource allocation would be based on priority groups (1-3) with ties within groups broken by comorbidities known to affect short-term recovery, then age (ie, younger patients receiving priority), followed by provision of an essential function within health care, then actual priority score (1-8), and, finally, lottery. If we were ever to implement this process, all allocation decisions would be made by a triage team consisting of the chief medical and nursing officers or designees, a critical care physician, an ethicist, and 1 person each from nursing or social work leadership. We recommended consideration be given to including a person with a disability and a member of the clergy. While our policy is not publicly available, it is the basis for a policy approved by the Florida Bioethics Network, the Florida Developmental Disability Council, and the Florida Hospital Association.¹¹

To prepare for possible resource allocation need, we aimed to assess the feasibility of rapidly calculating daily priority scores for all patients at risk of potential ventilator allocation (to or away from such support). A team of 9 third- and fourth-year medical students were recruited and trained on how to calculate SOFA scores and how to review the electronic medical record for evidence of relevant comorbidities. A scoring how-to guide was created to enhance the likelihood that all students collected data similarly (eFigure 1 in the Supplement). Each day, students calculated scores for all relevant patients (each patient’s daily score was calculated by a single student) and entered them into a daily log in Excel 2013 (Microsoft Corp), which was kept on a secured Health Insurance Portability and Accountability Act–protected cloud-based server. At the tertiary hospital, the SOFA scores were automatically calculated by the electronic health record using an algorithm built locally and validated with medical record review prior to use. At the quaternary hospital, SOFA scores were calculated manually by students.

We included all patients entered into the QI data set in the cohort study. Patients were selected for inclusion in the QI project if they were admitted to an adult COVID-19 unit (ICU and non-ICU) at either hospital. At the tertiary hospital, we also included patients without COVID-19 who were admitted to an ICU or intermediate care unit and who were currently receiving mechanical ventilation (invasive or noninvasive) or high-flow nasal cannula considering that any ventilator allocation would apply to patients with and without COVID-19; patients outside COVID-19 units were not included at the quaternary hospital due to QI project–related resource limitations. We excluded patient-days without available SOFA scores and patients without comorbidity data (eFigure 2 in the Supplement). All analyses were done with the observation at the level of the patient (not patient-day).

We considered race (ie, White, Black, Asian, or multiracial) and ethnicity (Non-Hispanic or Hispanic) as separate exposures. Both exposures were taken from information provided in the electronic medical record, which is based on patient self-identification (or surrogate input if patients were not able).

We evaluated 2 coprimary outcomes (ie, maximum and minimum priority score [1-8]) for each patient across all available patient-days of data. We chose the priority score (rather than priority group) to allow for a more granular analysis and because these scores would be used to break ties within priority groups. We considered the maximum and minimum score from each patient’s daily scores across the study period because the high or low scores for each patient would be likely to determine whether they would be denied (maximum score) or would receive (minimum score) access to resources. Secondary outcomes included both maximum and minimum priority groups, SOFA scores, and SOFA points. How tiebreakers (eg, comorbidities affecting short-term recovery, age, essential worker status) would affect triage was not evaluated.

We described the cohort using standard summary statistics and compared characteristics across groups using χ² and Kruskal-Wallis testing, as appropriate. To assess the independent associations of race and ethnicity with triage priority, we created a series of 8 multivariable Poisson regression models, 1 for each outcome (eg, maximum priority score). Potential confounders for both exposures were considered similar and were all included in each model, as follows: sex (male or female), preferred language (English, Spanish, or other), median income of home zip code (<$25 000, $25 000 to <$50 000, $50 000 to <$75 000, or ≥$75 000), primary insurer (Medicare/Medicaid, commercial, or none), age, admission to a COVID-19 ward, and hospital (tertiary or quaternary). Because each exposure (race and ethnicity) was considered a potential confounder for the other, a single model including both exposures was constructed to assess each exposure’s association with each outcome. Our primary models included complete cases; we conducted a sensitivity analysis including all patients and including an unknown category for all covariates. We conducted a second pair of sensitivity analyses excluding covariates that may track with race or ethnicity and are actually components of structural racism (ie, median income and primary insurance).

To determine whether including information about longer-term prognosis (ie, comorbidity points) was associated with the prioritization of patients of different races and ethnicities, we compared patient prioritization by SOFA points alone (categorized in 3 groups) vs priority groups (based on SOFA points plus comorbidities). Quantification of the association of including comorbidities was assessed as the proportion of patients in each race and ethnicity group who achieved higher or lower priority after comorbidity inclusion.

All statistical analyses were performed using Stata 16 (StataCorp) and Excel 2013 (Microsoft Corp). A 2-tailed P < .05 was considered statistically significant; no adjustment was made for multiple comparisons.

The cohort was composed of 1127 patients (675 [59.9%] from the tertiary hospital; median [interquartile range {IQR}] age, 62.7 [51.7-73.7]; 607 [53.9%] men) and 5613 days of data (3296 [58.7%] from the tertiary hospital). Overall, 711 (63.1%) were White patients, 323 (28.7%) were Black patients, 8 (0.7%) were Asian patients, 31 (2.8%) were multiracial patients, and in 54 patients (4.8%), race was unknown; 480 (42.6%) were Non-Hispanic patients, 611 (54.2%) were Hispanic patients, and 36 (3.2%) had unknown ethnicity.

A total of 782 patients (69.4%) had a maximum priority group assignment of 1, while 255 (22.6%) were in group 2, and 90 (8.0%) were in group 3 (Table 1 and Figure 2). The median (IQR) maximum priority score for the cohort was 3 (1-4); the median (IQR) minimum score was 2 (1-3). Patients in maximum priority group 3 were more likely to be older (median [IQR] age: group 3, 68.5 [55.0-79.0] years; group 2, 66.3 [57.1-75.8] years; group 1, 61.0 [50.1-70.9] years; P < .001) with more comorbidities (those with reduced 5-year survival: group 3, 55 [61.1%]; group 2, 147 [57.6%]; group 1, 206 [26.3%]; P < .001; those with reduced 1-year survival: group 3, 70 [77.8%]; group 2, 147 [57.6%]; group 1, 0; P < .001). Patients with a maximum priority group of 3 were less likely to be admitted to a COVID-19 ward (group 3, 36 [40.0%]; group 2, 113 [44.3%]; group 1, 541 [69.2%]; P < .001); however, patients being cared for in a COVID-19 ward may have been admitted to general medical units while patients not receiving care in a COVID-19 ward were only admitted to ICUs or intermediate care units. Similar associations were found with minimum priority groups (eTable 2 in the Supplement).

There were no significant differences in maximum priority group across races (White patients: group 1, 500 [63.9%]; group 2, 156 [61.2%]; group 3, 55 [61.1%]; Black patients: group 1, 227 [29.0%]; group 2, 71 [27.8%]; group 3, 25 [27.8%]; P = .25) or ethnicities (Hispanic patients: group 1, 440 [56.3%]; group 2, 128 [50.2%]; group 3, 43 [47.8%]; P = .22). Similarly, no significant differences were found in race and ethnicity breakdowns across minimum priority groups.

After multivariable adjustment, there was no association of race with maximum priority score using White patients as the reference group (Black patients: incidence rate ratio [IRR], 1.00; 95% CI, 0.89-1.12; Asian patients: IRR, 0.95; 95% CI, 0.62-1.45; multiracial patients: IRR, 0.93; 95% CI, 0.72-1.19) or ethnicity using non-Hispanic patients as the reference group (Hispanic patients: IRR, 0.98; 95% CI, 0.88-1.10) (Table 2). Similarly, no association was found with minimum priority score using the same reference racial and ethnic reference groups (Black patients: IRR, 1.01; 95% CI, 0.90-1.14; Asian patients: IRR, 0.96; 95% CI, 0.62-1.49; multiracial patients: IRR, 0.81; 95% CI, 0.61-1.07; Hispanic patients: IRR, 1.00; 95% CI, 0.89-1.13). The only association found between self-identified race or ethnicity across any secondary outcomes was for maximum SOFA score, for which multiracial patients (compared with White patients) were more likely to have a higher SOFA score (IRR, 1.33; 95% CI, 1.12-1.59; P = .001) (eTables 3-5 in the Supplement). In the sensitivity analyses using the full cohort and assigning missing data to an unknown category (eTable 6 in the Supplement) and removing socioeconomic factors as covariates (eTable 7 and eTable 8 in the Supplement), results were qualitatively the same.

When comparing maximum priority group (based on SOFA plus comorbidity) information to triage groups based on maximum SOFA points alone, 10% of the cohort would receive higher and 16% lower priority for resource allocation with the inclusion of comorbidity data (Figure 3). This change in prioritization was similar for White patients (10% higher, 16% lower) and Black patients (8% higher, 16% lower). Asian patients (25% higher, 13% lower) and multiracial patients (19% higher, 9% lower) appeared to move into higher priority groups at greater rates than other groups with the inclusion of comorbidities. Inclusion of comorbidities resulted in Hispanic patients receiving higher prioritization 10% of the time (11% for non-Hispanic patients) and lower prioritization 14% of the time (20% for non-Hispanic patients). Comparable relative rates of reprioritization across races and ethnicities were seen when considering minimum priority group vs minimum SOFA point–based group (eFigure 3 in the Supplement).

As hypothesized, we found no association of race or ethnicity with either maximum or minimum priority score. Across 6 secondary outcomes, the only significant association identified was self-identification as a multiracial person (compared with White) with an increase in maximum SOFA score but not SOFA points. This finding is of no consequence for resource allocation because our CSC protocol used SOFA points, not SOFA score. Additionally, despite concerns that inclusion of comorbidity information would lead to deprioritization of individuals from underrepresented minority groups, the priority groups assigned to Black and White patients were similarly affected by the addition of comorbidity data. Asian and multiracial patients as well as those with Hispanic (vs non-Hispanic) ethnicity fared relatively better with the inclusion of comorbidity data.

There is good reason to be concerned that COVID-19–related CSC policies may negatively affect racial and ethnic minorities. Disparities have been identified in relation to COVID-19; test positivity rates, hospitalization, and, in some studies, mortality rates are higher among Black^12-23 and Hispanic individuals.^13-16,22,23 Moreover, prior work has demonstrated that seemingly race/ethnicity–agnostic scoring systems may disadvantage minority patients. e.g., Vigil et al^24,25 found that being non-Hispanic Black or Hispanic (vs non-Hispanic White) was associated with being assigned a lower emergency severity index score on emergency department presentation.

There are several potential explanations for our findings that neither race nor ethnicity were associated with triage prioritization using our CSC policy. First, it is possible that there truly exists no association between race or ethnicity and triage priority when assigned using a composite of estimated short-term and longer-term survival. Evidence for higher comorbidity burdens among individuals from underrepresented minority groups is robust^26,27 and has been the focus of many concerns regarding possible disparities related to CSC policies.^3,8,9 There is also evidence that acuity of non–COVID-19 illness on ICU presentation²⁸ and COVID-19–related lung involvement on hospital admission²⁹ may be higher for individuals from racial/ethnic minority groups. However, our strategy of assigning a value only for the single most serious comorbidity a patient has and of grouping SOFA scores within broader buckets may have blunted some of these differences. It should be noted that the cohort included only patients after admission to a hospital. Race/ethnicity–associated differences in rates and timing of seeking hospital-based care and rates of hospital admission after presenting with COVID-19 may bias our findings. Second, our sample size may have been insufficient to identify a true association of race or ethnicity with triage priority. However, the relatively narrow confidence intervals surrounding the association of both Black (vs White) and Hispanic (vs non-Hispanic) patients with triage scoring strengthens our findings. Finally, our results may be affected by residual confounding, specifically socioeconomic factors. We used median income of a patient’s zip code and primary insurer to account partially for these influences, yet this adjustment is assuredly insufficient.

To our knowledge, ours is the first analysis to evaluate the association of race and ethnicity with a CSC policy during COVID-19. Its main strength stems from our diverse cohort, inclusive of more than 25% Black and more than 50% Hispanic patients. Additionally, this study allowed us to demonstrate that our scoring algorithm was successful in achieving score distribution across the cohort, a necessary step for any triage tool.

Our analysis has several limitations. First, longer-term survival was based on comorbidities identifiable from the electronic health record of each hospital. With differing access to care³⁰ and potentially different hospital admission patterns, it is possible that comorbidities were underdiagnosed and, potentially, underdocumented for certain racial and ethnic subgroups. Moreover, medical students were tasked with abstracting comorbidity information, and their knowledge and experience may have affected accuracy. However, use of diagnoses available in the electronic health record simulates the process we would use in real-time were resource allocation triage needed. Second, assignment of a triage priority score is only the first step in the process of resource allocation. Factors that would be used in practice to break ties among patients in the same priority group were not considered; however, it is possible that inclusion of these factors might actually mitigate against bias because younger populations³¹ and health care workers³² are disproportionately from minority racial/ethnic groups. Moreover, ultimate triage decisions would be made by a separate triage team. Whether unintended bias would enter this latter portion of triage decision-making was not evaluated in our study; however, the separate triage team would be masked to patients’ race and ethnicity. Third, while Black and Hispanic patients were well represented in the cohort, we had few patients from other racial groups. Fourth, our study did not consider disability status because such information was not available at the time of data analysis. Fifth, the cohort consisted of patients admitted to 2 academic hospitals in Miami, a city with a diverse population and medical staff; the external generalizability of our findings to other settings is unknown. Additionally, the impact of collaboration between regional hospitals and triage across them was not considered. Similarly, our work may not be generalizable to health systems with different triage policies (eg, those that give lower priority to patients with greater numbers of comorbidities). Sixth, although neither hospital experienced a lack of access to ventilators, other aspects of care (eg, medication availability, admission of higher acuity patients to intermediate care units instead of ICUs) certainly deviated from standards of care during this time; whether this affected triage scoring is unknown but, unfortunately, reflects the reality of care during a crisis when such triage may be necessary. Seventh, race and ethnicity were obtained from the electronic health record; misclassification based on erroneous race or ethnicity assignment as well as the intrinsic challenges associated with asking people to self-identify into racial and ethnic categories may have introduced bias.³³

In this cohort study of adult patients admitted to a COVID-19 unit at 2 US hospitals, there was no association of race or ethnicity with the priority score underpinning a resource allocation policy. The COVID-19 pandemic is a stark reminder of how unfair our society can be. Racial and ethnic minority groups have endured a disproportionate brunt of the disease and its consequences in the United States. Clinicians, hospital administrators, and governmental leaders have an obligation to minimize, and not exacerbate, such disparities. At the same time, the need to employ CSC amid a global pandemic cannot be ignored. The findings of this study that such a policy, based on both short-term and longer-term expected survival, did not appear to unintentionally disadvantage patients from underrepresented minority groups is reassuring. However, in the event that any such policy is activated, ongoing vigilance for evidence of such disparities will be essential and should be included in the implementation of any CSC policy.

Accepted for Publication: February 7, 2021.

Published: March 19, 2021. doi:10.1001/jamanetworkopen.2021.4149

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Gershengorn HB et al. JAMA Network Open.

Corresponding Author: Hayley B. Gershengorn, MD, Division of Pulmonary, Critical Care, and Sleep Medicine, Department of Medicine, University of Miami Miller School of Medicine, 1600 NW 10th Ave, Rosenstiel Medical Science Building, Room 7043B, Miami, FL 33136 (hbg20@med.miami.edu).

Author Contributions: Dr Gershengorn had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Gershengorn, Holt, Mora, West, Goodman, Kett, Brosco.

Acquisition, analysis, or interpretation of data: Gershengorn, Holt, Rezk, Delgado, Shah, Arora, Colucci, Mora, Iyengar, Lopez, Martinez, West, Kett, Brosco.

Drafting of the manuscript: Gershengorn, Holt, Delgado, Mora, Iyengar, Goodman.

Critical revision of the manuscript for important intellectual content: Gershengorn, Holt, Rezk, Shah, Arora, Colucci, Lopez, Martinez, West, Goodman, Kett, Brosco.

Statistical analysis: Gershengorn, Rezk, Iyengar.

Administrative, technical, or material support: Holt, Rezk, Delgado, Shah, Mora, Lopez, Kett, Brosco.

Supervision: Lopez, Kett.

Communicating crisis standards of care to institutional leadership: Goodman.

Conflict of Interest Disclosures: Dr Gershengorn reported receiving personal fees from Gilead outside the submitted work. No other disclosures were reported.

Funding/Support: Dr Gershengorn received funding from the University of Miami Hospital through the UHealth-DART research group.

Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.