Clinical Severity Prediction Scores in Low-Resource Settings and the Conundrum of Missing Data

The ability to predict decompensation and mortality owing to sepsis has been an elusive goal for clinicians in both high- and low-resource settings for many years. Identifying patients at risk of decompensation can lead to more accurate counseling of patients and families, targeted allocation of limited resources, and better comparison of research populations. Specific focus on improved risk stratification of patients with sepsis in Africa is urgent—whereas nearly 20% of all global deaths are associated with sepsis each year, the largest portion of this burden has fallen to the continent of Africa.¹ Given the tremendous burden of sepsis in sub-Saharan Africa and limited medical resources in some settings, the identification of clinical scores to assist in the early and accurate identification of patients at greatest risk of a poor outcome is of great interest.

Prognostic scores frequently used in critical care, such as the Acute Physiology and Chronic Health Evaluation and the Simplified Acute Physiology Score, have proven effective in high-resource settings but not in low-resource settings.² Some reasons for this difference are the excessive burden of resources such as invasive monitoring devices and the volume of data required. In addition, the heterogeneity of patient populations, underlying infections, and clinical environments across low-resource settings has further impeded efforts to accurately predict those who may decompensate or die during their hospitalization. With these points in mind, several authors have attempted to create and/or validate scores specific to patients in low-resource settings.^3,4

Bonnewell and colleagues⁵ report the results of their prospective assessment of the ability of several clinical illness severity scores to predict in-hospital death among febrile adults admitted to 2 hospitals in Tanzania over a 2.5-year period. Participants underwent a standardized clinical history and physical examination as well as a battery of laboratory testing including HIV antibody testing and blood cultures. The authors focused primarily on the Universal Vital Assessment (UVA) score,³ which was developed using cohorts from sub-Saharan Africa. They then compared the UVA score performance with that of the Modified Early Warning Score (MEWS), National Early Warning Score (NEWS), quick Sepsis Related Organ Failure Assessment (qSOFA) score, and Systemic Inflammatory Response Syndrome (SIRS) criteria.

The 597 participants in the study were similar to those in other cohorts of hospitalized febrile adults in sub-Saharan Africa, with a median age of 43 years (IQR, 31-56 years) and a 33.2% HIV positivity rate.^3,6 The in-hospital mortality was 9.2% overall and 17.7% for patients positive for HIV; given the strong association of HIV status with mortality, the authors presented baseline characteristics and several results among both the full cohort and the subset of patients positive for HIV. Of note, given the study’s standardized assessment of all enrolled patients, only 21 participants (3.4%) were excluded because data required to calculate the various clinical scores were missing.

Overall, the authors concluded that the UVA score best predicted hospital mortality among the full cohort. Those with the highest vs lowest UVA scores (≥5 vs ≤1 points) had the largest risk of death of any score group (risk ratio [RR], 30.6; 95% CI, 9.6-97.8), followed by those with a high vs low qSOFA score (≥2 vs 0 points; RR, 15.4; 95% CI, 3.8-63.1). The UVA score had the highest area under the receiver operating characteristic curve of all scores evaluated (0.85); others ranged from 0.64 to 0.81.

This study’s prospective nature and multicenter design are strengths, and it is a laudable follow-up to a previous study proposing the UVA score.³ The authors have contributed important new findings to the evolving search for a mortality prediction tool with both good discrimination and generalizability to resource-constrained settings. The fact that all clinical and laboratory data contributing to the exposure variables were collected prospectively by the study team instead of relying on clinically available data is both a strength and a weakness. This design and the rigor with which the study was carried out led to impressively low rates of missing data, which allowed the scores to be evaluated in near-perfect circumstances. Therefore, the reader can conclude that if all data elements of a chosen score will be collected for every febrile hospitalized adult, they might expect to find similar score performance among a comparable patient population. Given the near-perfect data collection, it is unsurprising that one of the most detailed scores, requiring 7 data points (both the UVA score and NEWS require 7 variables, compared with 3, 4, and 5 for the qSOFA score, SIRS criteria, and MEWS, respectively), had the best performance. However, this study design makes it nearly impossible to understand how the scores actually compare in clinical practice in low-resource settings, where even basic clinical data such as vital signs and complete blood cell count results are frequently missing.^2,3,6 Haniffa and colleagues have previously noted that a major challenge in the use of mortality prediction tools developed in high-resource settings is the volume of missing data.² Indeed, the initial study³ proposing the UVA score using a secondary analysis of multiple sub-Saharan African cohorts found that nearly 50% of necessary data were missing—this is likely a more accurate reflection of many low-resource settings.

Further work to refine the UVA score and other illness severity tests will be necessary to develop a tool, or perhaps a set of tools for different clinical environments, that can be easily implemented with minimal missing data in a real-world setting and that have good prognostic accuracy. The strong performance of the UVA score in the study by Bonnewell et al⁵ suggests that evaluation of how to further improve its performance may be helpful. In addition, a direct comparison of the UVA score with the Tropical Intensive Care Score,⁴ which was not included in the study by Bonnewell et al,⁵ would offer useful data. The Tropical Intensive Care Score has been shown to have good score performance compared with the Acute Physiology and Chronic Health Evaluation II and Simplified Acute Physiology Score II among 3855 critically ill patients in 18 South Asian intensive care units, but we are unaware of any studies directly comparing it with the UVA score.

Although previous studies have made it clear that clinical scores designed in resource-rich settings are often not useful in resource-constrained environments, it remains unclear which alternatives are best. More research in this area is needed, especially across levels of resource availability within low- and middle-income country settings. In the meantime, clinicians working in resource-constrained settings will continue to advocate for resources and systems of care that allow for the provision of basic, appropriate physical examination and laboratory testing for every ill hospitalized patient, regardless of inclusion in research studies.

Published: December 16, 2021. doi:10.1001/jamanetworkopen.2021.37593

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Brotherton BJ et al. JAMA Network Open.

Corresponding Author: Kristina E. Rudd, MD, MPH, Department of Critical Care Medicine, University of Pittsburgh School of Medicine, 3520 Fifth Ave, Ste 100, Pittsburgh, PA 15213 (ruddk@pitt.edu).

Conflict of Interest Disclosures: None reported.