Clinical Prognostic Rules for Severe Acute Respiratory Syndrome in Low- and High-Resource Settings

Background  An accurate prognostic model for patients with severe acute respiratory syndrome (SARS) could provide a practical clinical decision aid. We developed and validated prognostic rules for both high- and low-resource settings based on data available at the time of admission.

Methods  We analyzed data on all 1755 and 291 patients with SARS in Hong Kong (derivation cohort) and Toronto (validation cohort), respectively, using a multivariable logistic scoring method with internal and external validation. Scores were assigned on the basis of patient history in a basic model, and a full model additionally incorporated radiological and laboratory results. The main outcome measure was death.

Results  Predictors for mortality in the basic model included older age, male sex, and the presence of comorbid conditions. Additional predictors in the full model included haziness or infiltrates on chest radiography, less than 95% oxygen saturation on room air, high lactate dehydrogenase level, and high neutrophil and low platelet counts. The basic model had an area under the receiver operating characteristic (ROC) curve of 0.860 in the derivation cohort, which was maintained on external validation with an area under the ROC curve of 0.882. The full model improved discrimination with areas under the ROC curve of 0.877 and 0.892 in the derivation and validation cohorts, respectively.

Conclusion  The model performs well and could be useful in assessing prognosis for patients who are infected with re-emergent SARS.

Severe acute respiratory syndrome (SARS) was a novel disease that infected more than 8000 people in 29 territories between November 2002¹ and July 2003. Approximately 10% of infected patients worldwide died,² although the case fatality ratio varied by site.³ There is a real possibility that SARS will re-emerge based on the likely existence of at least 2 known animal reservoirs in civets and horseshoe bats that are commonly found in the wild as well as reared and consumed in southern China.^4-6

Prognostic models have long been accepted as useful decision aids for clinicians and have previously been developed for use in infectious diseases such as influenza⁷ and pneumonia,^8,9 as well as a variety of chronic conditions.^10-12 In the event of another SARS outbreak, clinicians may find useful an easy-to-apply prognostic rule that can quantify the risk of mortality early in the course of illness.

We describe the derivation (Hong Kong cohort) and validation (Toronto, Ontario) cohort of 2 models that predict prognosis in patients with “probable SARS” based on information available at the time of hospital admission. We developed 2 models (basic and full), one for use in low-resource and the other in high-resource settings, because if SARS were to re-emerge, it may well happen initially in a low-resource setting where the health care infrastructure is suboptimal, as it did in rural Foshan of Guangdong province in China during November 2002.¹ The full model for high-resource settings would apply to advanced health care systems.

We analyzed an integrated database containing clinical and epidemiologic details on all patients reported to have “probable SARS” according to the World Health Organization definition,¹³ who were hospitalized throughout the epidemic in Hong Kong.¹⁴ More detailed clinical and laboratory information for a subset of patients (ie, all consecutive patients with probable SARS from 2 large treatment hospitals) were abstracted from a standardized medical chart review by trained nurses and medical officers.

For the Toronto data, the Canadian SARS Research Network conducted a retrospective cohort study of all suspect or probable SARS cases among adults (age ≥16 years) admitted to a hospital during the outbreak. Medical charts were reviewed by trained research nurses, and then clinical, radiographic, and laboratory data were double entered into a database.

Following a literature review^14-28 to identify key variables that were likely to be most important in predicting death, we considered the following for inclusion in the prognostic decision rule: age, sex, pre-existing comorbid conditions (including asthma, chronic obstructive pulmonary disease, cardiovascular disease, cerebrovascular disease, cancer, diabetes mellitus, chronic renal disease, and chronic liver disease), chest radiograph on admission (normal, haziness, or unilateral or bilateral infiltrates), oxygen saturation on room air, lactate dehydrogenase (LDH) level on admission, neutrophil count, platelets, and lymphocyte count. For the Hong Kong sample, data on LDH were available for most patients, but the other measures including chest radiograph, oxygen saturation, and white blood cell components were only available for the subset of patients as indicated previously.

We derived 2 separate prognostic models, a basic model using background information including age, sex, and comorbid conditions and a full model that further incorporated radiologic and laboratory results on admission to a hospital. We developed a score-based prognosis model by using multivariable logistic regression to identify the strength of association between the different predictors at the time of admission and case fatality due to SARS.²⁹ Integer scores for each predictor variable were generated by dividing the estimated β coefficients by the absolute value of the β coefficient with the smallest magnitude and then rounding to the nearest integer. For both the basic and full models, the overall risk score was calculated by summing each component together. Using the estimated intercept terms of each multivariable logistic regression model, we created a table of the probability of death due to SARS for particular scores, as estimated by each model.

We derived each prognostic model using all available information on patients from Hong Kong.³⁰ We used multiple imputation^31,32 (10 imputations) in the derivation process to make the most of all available nonmissing data while preserving the uncertainty from the missing data in the results.³³ During the modeling process, we investigated the shape of the relationships of the continuous clinical and laboratory variables, and we condensed the comorbidity and chest radiograph variables into fewer groups. Further details on the modeling process are given at http://www.hku.hk/cmd/bjc/SARS_CPR_appendix.pdf.

To investigate the risk stratification provided by each prognosis model, patients were categorized by quartiles of total risk score, and the observed case-fatality ratios were computed in each quarter. To quantify the discrimination of each prognosis model, the area under the receiver operating characteristic (ROC) curve was calculated.^30,34 Confidence intervals for the areas under ROC curves were calculated by bootstrapping with 1000 repetitions.³⁵

We internally validated the prognostic models using a bootstrap resampling approach to calculate an optimism-corrected (“optimism” refers to the absolute magnitude of bias) estimate of the area under the ROC curve.^29,30,32 This procedure estimated the degree to which the predictive accuracy of the model would be expected to deteriorate when applied to an independent sample of patients.

To explore the possibility of treatment bias affecting the results in terms of the use and timing of ribavirin and corticosteroids, the discrimination of the prognosis models was estimated separately in treatment subgroups. We also validated the performance of the prognostic models in the subset of the Hong Kong cohort with laboratory confirmation of SARS.³⁶ Finally, the models were validated using the secondary outcome of intensive care unit admission or death. Because not all of the patients with probable SARS had laboratory confirmation of SARS infection and may therefore have had a milder disease, we compared our rule with one derived using only the subset of the Hong Kong cohort with laboratory confirmation of SARS.

The prognosis models were externally validated on data from Toronto. Similarly, multiple imputation was used to impute any missing data, and the prognosis models were applied to these patients, allowing the performance of the model to be assessed in another setting. The prognostic models were further validated on only those patients from Toronto who had laboratory confirmation of SARS. We conducted all analyses using R version 2.1.0.³⁷

This study received ethics approval from the institutional review boards of the University of Hong Kong/Hospital Authority Hong Kong West Cluster, Hong Kong; McMaster University, Hamilton, Ontario; the University of Toronto, Toronto; and all Toronto-area hospitals where data were collected.

The Hong Kong derivation cohort comprised 1755 patients with probable SARS, 302 (17.2%) of whom died. Baseline demographic and clinical characteristics of these patients are given in Table 1.

β Coefficients of the multivariable logistic regression models applied to the derivation cohort, and associated integer scores are presented in Table 2. In the basic model, the assigned scores were derived by dividing all regression coefficients by a factor of 0.239 and rounding to the nearest integer. In the full model, the divisor was 0.095. In each model, the total score, summing over all the predictor variables, can be converted to an estimate of the probability of death as given in Table 3. For example, a male patient aged 45 years with no comorbid conditions would score 7 under the basic model (Table 2), corresponding to a probability of death of 14% (Table 3).

The performance indexes of the basic prognostic model on the derivation and validation cohorts are summarized in Table 4. The area under the ROC curve in the Hong Kong derivation cohort was 0.860 (95% confidence interval, 0.837-0.884). In the internal validation procedure we estimated that the optimism-corrected area under the ROC curve was 0.859. Validation on the 1467 patients (84%) with laboratory confirmation of SARS gave an area under the ROC curve of 0.842.

The full prognostic model had higher discrimination on both the derivation and validation cohorts (Table 4). The area under the ROC curve in the Hong Kong derivation cohort was 0.877 (95% confidence interval, 0.854-0.900), which was reduced to 0.864 on internal validation and 0.863 on the patients with laboratory confirmation of SARS infection.

As a further validation exercise, the scores were assessed on the alternative outcome of intensive care unit admission or death. The basic and full models achieved good discrimination on this outcome (areas under the ROC curve of 0.790 and 0.809, respectively).

To check for possible effect modification between the discrimination of the prognostic models and the different treatment regimens, the Hong Kong cohort was split into 4 subgroups based on treatment with ribavirin or corticosteroids within 7 days of symptom onset. The 4 subgroups comprised patients who were not treated with these drugs, treated with ribavirin only, treated with corticosteroids only, or treated with both. The areas under the ROC curve in the 4 subsets were consistent with the estimated areas under the ROC curve of the prognosis models in the full cohort (Table 4). The results were similar when we categorized patients into treatment groups based on initiation within 3 or 5 days of symptom onset and within 3, 5, or 7 days of hospital admission (data not shown).

As a sensitivity analysis, we derived a rule using only data from the 1467 patients with laboratory-confirmed SARS and found that all the estimated regression coefficients were similar to those presented in Table 3, except that the effect of LDH was somewhat attenuated and the effect of neutrophilia was increased (data not shown).

The Toronto validation cohort comprised 291 patients with probable SARS, 41 (14.1%) of whom died. The prognostic models were applied to these data, and the areas under the ROC curve were estimated to be 0.882 for the basic prognosis model and 0.892 for the full model. Validation was also conducted on the 208 patients (71.5%) with laboratory confirmation of SARS infection, and in these patients the areas under the ROC curve were 0.868 and 0.888 for the basic and full models, respectively. The limited sample size prohibited the evaluation of the prognosis models on treatment-stratified samples in the Toronto validation cohort.

In addition to good discrimination, both prognostic indexes had good predictive performance. In the basic model, stratification by quartiles of the index (0, 1-3, 4-9, and 10-21) revealed a positive and exponentially increasing gradient in the risk of death as shown in the Figure, A. The proportion of patients who died in the 4 quarters were 2.3%, 4.8%, 11.8%, and 53.8%, respectively, which correlate well with the predicted probabilities (Table 3) for the ranges of scores in each quarter. The full model was also well calibrated, as shown in the Figure, B. We found that good predictive performance of our prognostic models was maintained in the Toronto cohort (Figure, C and D), although the observed event rates were slightly lower than predicted.

Our findings suggest that a simple prognostic model using only data on patient history can be used to accurately predict the prognosis of a patient with SARS and provide a practical decision aid in low-resource settings. With the additional radiologic and laboratory data, the full prognosis model provides even more accurate estimates of patient prognosis. Both prognostic models achieved high areas under the ROC curves in the Hong Kong derivation cohort, which were maintained on internal validation by bootstrapping. The generalizability of both models was confirmed with high areas under the ROC curves on external validation using data from Toronto.

We further note the generalizability of our decision rule to all patients infected by the SARS coronavirus beyond those captured in the case series in both Hong Kong and Toronto examined here. A recent meta-analysis of 16 seroprevalence (of SARS coronavirus IgG) studies yielded an overall positivity rate of 0.10% (95% confidence interval, 0.02%-0.18%) in asymptomatic contacts of SARS cases or the general population³⁸; this means that infection with SARS coronavirus invariably caused severe disease requiring hospitalization. Thus, our findings based on complete samples of hospitalized SARS cases in 2 different countries should be applicable to all SARS-infected patients.³⁹ The generalizability of the prognostic model to other settings is also supported because the clinical manifestations of SARS were somewhat different in the 2 settings; for example, gastrointestinal symptoms were more common in Hong Kong¹⁵ than in Toronto.¹⁶

In a symptom-based diagnostic model derived for SARS infection among patients presenting at an emergency department, the diagnostic rule used a specific cutoff to categorize patients as SARS or non-SARS.⁴⁰ The authors could then investigate the sensitivity and specificity of their diagnostic rule, based on the post hoc assessment of whether each patient had SARS according to laboratory results. For our prognostic model, we have chosen not to dichotomize patients into a high- or low-risk group, but rather have used the rule to generate specific probabilities of death.

The most important factor in predicting prognosis was age (Table 2). The next most predictive variables were the presence of 2 or more comorbid conditions and neutrophilia. Whereas the odds ratios of death for any characteristic can be easily calculated from Table 2 by taking the antilog of the regression coefficient, the odds ratio for patients with high neutrophil counts vs those with reference levels was 2.49 (exp[0.914]). This finding is consistent with previous analyses of subgroups of patients with SARS from Toronto¹⁶ and Hong Kong,^26,27 which found strong positive associations between neutrophilia and death. Elevated neutrophil counts are associated with severe sepsis, stress, and systemic inflammation. Abnormal chest radiography results and low oxygen saturation levels were both indicators of more advanced or severe disease at admission and therefore carried a higher risk of death.^18-22 High levels of LDH, another proxy for illness severity, increased the risk of mortality, as would be expected.^14,24-28 Low platelet levels are typically associated with intravascular disturbance, which may explain the association with an increased risk of death; a similar observation had also been discussed in a study of patients with SARS in Beijing.²² Based on the present findings, information such as LDH, neutrophil, and platelet levels should be routinely considered as important prognostic markers in patients with probable SARS.

The results in Table 4 suggest that the performance of the prognostic model was unaffected by early treatment choice. In particular, the model maintained high discriminative power regardless of the treatment regimen administered. This important observation concerning the robustness of our rule suggests that the prognosis for a patient is not affected by treatment selection, conforming to the commonly held view during and after the epidemic that none of the widely used treatments were particularly effective.^14,41-45 However, we would certainly hope that in future epidemics we will have both new immunomodulating compounds as well as antiviral agents that will allow us to intervene therapeutically. The fact that the model is constructed based on our initial experience with this disease (in fact, our only experience to date) is an important but, fortunately, necessary limitation. Further assessment of the effectiveness of the various treatments used against SARS should be carried out as a priority.

Our analyses were based on patients with probable SARS according to the World Health Organization definition.¹³ We adopted this definition rather than laboratory confirmation because the latter definition may be potentially biased toward including more survivors, particularly in the early cases.¹⁴ Furthermore, rapid diagnostic SARS tests were not available until fairly late in the epidemic and had poor sensitivity for detecting the disease.^46-48

Our study has several potential limitations. First, the data were based on retrospective medical chart review and may not have included all available data about each patient but only the data that were recorded in the chart and subsequently abstracted. Second, the formulation of the full prognosis rule was based on clinical data at admission and thus ignored any potential changes in prognosis as a patient's condition changed over time. Further studies should investigate other potential prognostic models at different time points, such as 3, 7, and 14 days after admission (given the typical admission-to-discharge and admission-to-death intervals were 20-30 days¹⁴), using more detailed clinical parameters if available. Nevertheless, this study has shown that a good estimate of prognosis can be made at admission. Third, most of the clinical information included in the full prognosis model was based on data from a subset of the patients with SARS in Hong Kong. If the clinical characteristics of these patients differed systematically from the rest of the patients with SARS in Hong Kong, our estimates may have suffered from selection or inclusion bias. However, the methods we used allowed the best chance to minimize such bias,^31,33 and the likely lack of serious bias is further demonstrated by the excellent external validation of our rule. Fourth, for the sake of parsimony, the decision rule used predominantly dichotomous variables that may have oversimplified the way these predictor variables are sometimes interpreted clinically. Finally, although we have externally validated the prognostic models on data from Toronto, it would be useful to further investigate the transferability of the models to other countries or settings, especially those with a lower-reported case fatality ratio or where alternative treatment regimens were used, such as in mainland China and Taiwan. If those results confirm our findings, we can be further reassured that treatment bias was not a major threat to validity in our model. Similarly, if SARS were to re-emerge, it would be important to validate our model against illness caused by the new strain of the virus with a potentially different clinical manifestation.

Correspondence: Gabriel M. Leung, MD, MPH, Department of Community Medicine, University of Hong Kong, 21 Sassoon Rd, Pokfulam, Hong Kong (gmleung@hku.hk).

Accepted for Publication: December 23, 2005.

Author Contributions: Drs Cowling and Muller had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Financial Disclosure: None reported.

Funding/Support: This work was supported in part through a commissioned research grant from the Research Fund for the Control of Infectious Diseases of the Health, Welfare, and Food Bureau of the Hong Kong Special Administrative Region Government and by the University of Hong Kong SARS Research Fund. The Canadian SARS Research Network was funded by the Canadian Institutes for Health Research.

Acknowledgment: We thank our colleagues in the Department of Health and Hospital Authority involved with the public health control of the SARS epidemic and data collection and processing.

Additional Information: Statistical methodology can be accessed at http://www.hku.hk/cmd/bjc/SARS_CPR_appendix.pdf.