The area under the curve was similar for ABCD-10 and SCORTEN at both Day 0 (n = 334) (0.816; 95% CI, 0.759-0.872 vs 0.827; 95% CI, 0.774-0.879) (P = .72) (A) and after 48 hours (n = 287) (0.823; 95% CI, 0.760-0.885 vs 0.848; 95% CI, 0.787-0.909) (P = .56) (B). ABCD-10 indicates age, bicarbonate level, cancer, dialysis, and 10% involved body surface area; SCORTEN, Score of Toxic Epidermal Necrolysis.
Customize your JAMA Network experience by selecting one or more topics from the list below.
Noe MH, Rosenbach M, Hubbard RA, et al. Development and Validation of a Risk Prediction Model for In-Hospital Mortality Among Patients With Stevens-Johnson Syndrome/Toxic Epidermal Necrolysis—ABCD-10. JAMA Dermatol. 2019;155(4):448–454. doi:10.1001/jamadermatol.2018.5605
Can a novel prognostic model for risk of in-hospital mortality be developed for Stevens-Johnson syndrome/toxic epidermal necrolysis in a contemporary, multi-institutional cohort of patients from the United States?
A 5-item mortality prediction model (ABCD-10; age, bicarbonate level, cancer, dialysis, and 10% involved body surface area) was developed from 370 patients. The model achieved good discrimination and calibration for predicting in-hospital mortality, although its performance was not significantly different from that of the SCORTEN model (Score of Toxic Epidermal Necrolysis).
Among patients with Stevens-Johnson syndrome/toxic epidermal necrolysis, ABCD-10 accurately predicted in-hospital mortality; however, ABCD-10 requires validation in other cohorts to determine its generalizability.
Stevens-Johnson syndrome/toxic epidermal necrolysis (SJS/TEN) is a spectrum of severe mucocutaneous drug reaction associated with significant morbidity and mortality. A previously developed SJS/TEN-specific severity-of-illness model (Score of Toxic Epidermal Necrolysis [SCORTEN]) has been reported to overestimate and underestimate SJS/TEN-related in-hospital mortality in various populations.
To derive a risk prediction model for in-hospital mortality among patients with SJS/TEN and to compare prognostic accuracy with the SCORTEN model in a multi-institutional cohort of patients in the United States.
Design, Setting, and Participants
Data from a multicenter cohort of patients 18 years and older treated for SJS/TEN between January 1, 2000, and June 1, 2015, were obtained from inpatient consult databases and electronic medical record systems at 18 medical centers in the United States as part of the Society for Dermatology Hospitalists. A risk model was derived based on data from 370 of these patients. Model discrimination (calculated as area under the receiver operating characteristic curve [AUC]) and calibration (calculated as predicted vs observed mortality, and examined using the Hosmer-Lemeshow goodness-of-fit statistic) were assessed, and the predictive accuracy was compared with that of SCORTEN. All analysis took place between December 2016 and April 2018.
Main Outcomes and Measures
Among 370 patients (mean [SD] age 49.0 [19.1] years; 195 [52.7%] women), 54 (15.14%) did not survive to hospital discharge. Five covariates, measured at the time of admission, were independent predictors of in-hospital mortality: age in years (odds ratio [OR], 1.05; 95% CI, 1.02-1.07), body surface area (BSA) in percentage of epidermal detachment (OR, 1.02; 95% CI, 1.01-1.04), serum bicarbonate level below 20 mmol/L (OR, 2.90; 95% CI, 1.43-5.88), active cancer (OR, 4.40; 95% CI, 1.82-10.61), and dialysis prior to admission (OR, 15.94; 95% CI, 3.38-66.30). A severity-of-illness score was calculated by taking the sum of 1 point each for age 50 years or older, epidermal detachment greater than 10% of BSA, and serum bicarbonate level below 20 mmol/L; 2 points for the presence of active cancer; and 3 points for dialysis prior to admission. The score was named ABCD-10 (age, bicarbonate, cancer, dialysis, 10% BSA). The ABCD-10 model showed good discrimination (AUC, 0.816; 95% CI, 0.759-0.872) and calibration (Hosmer-Lemeshow goodness of fit test, P = .30). For SCORTEN, on admission, the AUC was 0.827 (95% CI, 0.774-0.879) and was not significantly different from that of the ABCD-10 model (P = .72).
Conclusions and Relevance
In this cohort of patients with SJS/TEN, ABCD-10 accurately predicted in-hospital mortality, with discrimination that was not significantly different from SCORTEN. Additional research is needed to validate ABCD-10 in other populations. Future use of a new mortality prediction model may provide improved prognostic information for contemporary patients, including those enrolled in observational studies and therapeutic trials.
Stevens-Johnson Syndrome/toxic epidermal necrolysis (SJS/TEN) is a spectrum of rare, severe mucocutaneous drug reaction associated with significant morbidity and mortality. Clinically, patients present with atypical targetoid lesions or generalized erythema progressing to full-thickness epidermal necrosis. Patients are classified by the extent of epidermal detachment, with SJS representing less than 10% body surface area (BSA); SJS/TEN overlap, 10% to 30% BSA; and TEN, greater than 30% BSA. In-hospital mortality in adults has been estimated to be between 12% and 40% based on the extent of epidermal detachment at presentation.1,2 A recent study in the United States, based on diagnosis codes from the Nationwide Inpatient Sample, reported mortality rates of 4.8% for SJS, 19.4% for SJS/TEN overlap, and 14.8% for TEN.3 Unfortunately, the rarity of the disease makes it difficult to collect generalizable information to understand patient prognosis.
Previously, our research group4 reported a multi-institutional cohort of 377 patients with SJS/TEN from 18 academic medical centers across the United States. Based on SCORTEN predicted mortality, 74 deaths were expected, but 54 were observed (standardized mortality ratio [SMR], 0.73; 95% CI, 0.54-0.92).4 To our knowledge, the only other multi-institutional evaluation of SCORTEN was performed using a cohort of European patients diagnosed with SJS/TEN between 2003 and 2005 from the European Registry of Severe Cutaneous Adverse Reactions to Drugs. In that cohort of 166 patients, the observed mortality was 19%, with an expected (SCORTEN-predicted) mortality of 17% (SMR, 1.12; 95% CI, 0.78-1.57).5 While SCORTEN remains broadly applicable, improvements in supportive care over time, regional differences in patient risk factors, and variation in physician treatment preferences may affect its prognostic ability. The objective of the present study was to derive a prediction model for in-hospital mortality among patients with SJS/TEN and to compare its prognostic accuracy with that of SCORTEN in a multi-institutional cohort of patients from the United States.
Creation of the multi-institutional US cohort was previously described in detail.4 Briefly, patients 18 years and older treated for SJS/TEN between January 1, 2000, and June 1, 2015, were identified from inpatient consult databases and electronic medical record systems at the participating institutions. The diagnosis of SJS/TEN was made by the consulting dermatology team at the time of initial presentation based on clinical and pathologic features. All diagnoses were subsequently confirmed by the dermatology hospitalist principal investigator at each site via detailed record review, based on predefined clinical data and histologic parameters and according to the consensus definition of SJS/TEN proposed by Bastuji-Garin et al.6 Detailed medical information, including relevant demographics, medications, medical comorbidities, clinical presentation (including laboratory data, physical examination, and suspected disease triggers), treatment regimen and timing, and outcome (morbidity and mortality), was abstracted from the medical record using a standardized data collection tool. All patients with information regarding mortality (n = 370) were included in this analysis. This study was approved by the University of Pennsylvania institutional review board, waiving written informed consent for deidentified data, and reported following the STROBE statement.7
Univariable logistic regression was performed for all variables, including the 7 binary variables included in SCORTEN. All variables significantly associated with in-hospital mortality with a significance level of P < .25 were included as candidates in the multivariable model. To improve efficiency and avoid instability in this model, given the relatively low number of events per variable, bootstrapping was used for model selection.8,9 We used backward model selection with a threshold of P = .05 for elimination from the model, and 1000 bootstrap samples were performed. Multicollinearity was examined between predetermined groups of variables by examining pairwise combinations of the following variables: infection/sepsis, history of stem cell transplant/cancer, and chronic kidney disease/dialysis/serum creatinine/serum blood urea nitrogen. There was evidence of collinearity when inclusion of one variable in the model tended to result in the exclusion of the other(s). In these cases, the variable with the strongest association with the outcome, as determined by univariable logistic regression, was kept, and the other variable(s) was/were removed. After collinear variables were removed, variables that were present in more than 70% of the bootstrap samples were included in the model. A total of 364(98.4%) of the 370 patients had complete data and were included in the final model.
Using the independent predictors of mortality, an updated severity-of-illness score was developed. Continuous variables (ie, age and BSA) were transformed to binary variables by creating a cut point that achieved the highest discrimination using a receiver operating characteristic (ROC) curve. The β coefficients from the logit model were used to create a simplified prognostic model. This logit model can be used to predict in-hospital mortality: Pr(death) = elogit/(1 + elogit) where logit = −3.764 + 0.898 (ABCD-10). Model performance was assessed with both discrimination and calibration, using the entire sample. Discrimination was assessed by calculating the area under the ROC (AUC), where an area of 1.0 suggests perfect discrimination, and 0.5 suggests that the model is equivalent to random guessing.10 ROC analysis was performed by comparing ABCD-10 with SCORTEN both at admission and 48 hours after admission because prior research has suggested the predictive value of SCORTEN may be best using values collected on day 3, or 48 hours after admission.11 Calibration was examined using the Hosmer-Lemeshow goodness-of-fit statistic,12 with which closer agreement between the observed and predicted outcomes produces a lower test statistic. A 2-component Brier score was also calculated to determine the overall accuracy of the predictions.13 A value of 0 suggests perfect accuracy.
There were 370 patients in the cohort with dermatologist-confirmed SJS/TEN and information regarding hospital mortality, most from 2010 onward (260/370, 70.0%). The cohort has been described in detail previously.4 In summary, 195 (52.7%) of the 370 patients were female, with a mean (SD) age of 49.0 (19.1) years. The median BSA on admission was 15.5% (interquartile range, 6%-30%). Fifty-four patients (15.1%) did not survive to discharge. SCORTEN was used to calculate the expected mortality for the US cohort (Table 1). As reported previously, 54 deaths were observed, though 74 were expected based on the SCORTEN-predicted mortality (SMR, 0.73; 95% CI, 0.54-0.92).4
In univariable analyses, age, BSA of denuded skin at the time of admission, serum creatinine level, chronic kidney disease, coronary artery disease, diabetes, active/ongoing cancer, active infection, sepsis, dialysis, and history of stem cell transplantation were all associated with a statistically significant increased risk of death (all data detailed in Table 1). Additionally, all prognostic factors from SCORTEN were associated with an increased risk of death, except for tachycardia of 120 bpm or higher and serum glucose level greater than 14 mmol/L. The significant variables, including those from the original SCORTEN, were included as candidates in the multivariable model.
In multivariable logistic regression, 5 factors were independently associated with in-hospital mortality when measured at the time of admission: age (in years), epidermal detachment (percentage of BSA), serum bicarbonate level, active/ongoing cancer, and dialysis (Table 2). Using these factors, an updated severity-of-illness score was created. This updated severity risk equation was named ABCD-10 to represent the prognostic factors included (age, bicarbonate, cancer, dialysis, and 10% BSA). The ABCD-10 score is calculated by taking the sum of factors present on admission: 1 point each for age 50 years or older, epidermal detachment greater than 10% BSA, and serum bicarbonate level below 20 mmol/L; 2 points for the presence of active/ongoing cancer; and 3 points for dialysis. The mortality associated with each ABCD-10 value is detailed in Table 3. Model diagnostics showed good discrimination (AUC, 0.816; 95% CI, 0.759-0.872) and good calibration (Hosmer-Lemeshow goodness-of-fit test, P = .30). The low Brier score (0.1484) confirmed excellent predictive ability of the model. Previous research has suggested that the predictive value of SCORTEN may be best using values collected on day 3, or 48 hours after admission,11 so mortality predicted by ABCD-10 was compared with the original SCORTEN at the time of admission and 48 hours later (Figure). The AUC was not significantly different for ABCD-10 compared with SCORTEN at both day 0 (0.816; 95% CI, 0.759-0.872 vs 0.827; 95% CI, 0.774-0.879; P = .72) and after 48 hours (0.823; 95% CI, 0.760-0.885 vs 0.848; 95% CI, 0.787-0.909; P = .56).
In a multi-institutional, contemporary cohort of 370 patients from the United States, 5 covariates, measured at the time of admission, were found to be independent predictors of in-hospital mortality: age in years (odds ratio [OR], 1.05; 95% CI, 1.02-1.07), epidermal detachment in percentage of BSA (OR, 1.02; 95% CI, 1.01-1.04), serum bicarbonate level below 20 mmol/L (OR, 2.90; 95% CI, 1.43-5.88), active cancer (OR, 4.40; 95% CI, 1.82-10.61), and dialysis prior to admission (OR, 15.94; 95% CI, 3.38-66.30). A severity-of-illness score was calculated by taking the sum of 1 point each for age 50 years or older, epidermal detachment greater than 10% of BSA, and serum bicarbonate level below 20 mmol/L; 2 points for the presence of active cancer; and 3 points for dialysis prior to admission. The score was named ABCD-10 (age, bicarbonate, cancer, dialysis, 10% BSA). The ABCD-10 model showed good discrimination (AUC, 0.816; 95% CI, 0.759-0.872) and calibration (Hosmer-Lemeshow goodness of fit test, P = .30). For SCORTEN, on admission, the AUC was 0.827 (95% CI, 0.774-0.879) and was not significantly different from that of the ABCD-10 model (P = .72).
SCORTEN is a well-established severity-of-illness score that, since its initial publication in 2000, has been used to predict SJS/TEN-related mortality. However, most previous studies were small, single-institution series reporting cases collected primarily before 2010.5,14-18 SCORTEN may be less accurate in critically ill patients with organ failure19,20 and in populations with different rates of medical comorbidities than those seen in the original single-center French cohort.21 Additionally, differences in treatment preferences by physicians and changes in supportive care over time may affect its accuracy in contemporary patients. In the United States, the majority of prior research examining the performance of SCORTEN comes from burn centers.22,23 The only previous study of patients under the care of dermatologists involved 24 patients treated at the University of Miami between 1993 and 1998.24 The authors reported an SMR of 0.91 (95% CI, 0.39-1.80), but the small number of patients and large confidence interval makes interpretation of the point estimate difficult.24 The findings from the present multicenter cohort suggest that SCORTEN may overestimate mortality in contemporary patients, and changes in the model may help improve prognostication. Such modifications to SCORTEN are critical given its importance as the reference standard for reported outcomes in SJS/TEN studies.
Previous studies examining SJS/TEN mortality have identified additional risk factors not present in the original SCORTEN model, likely owing to differences in the prevalence of underlying mortality risk factors in different patient populations. In 10 cases from India, tuberculosis was identified as an independent risk factor for mortality,21 while metabolic syndrome and/or gout were associated with increased mortality in 82 patients in Texas.23 Because of the rarity of the disease, much of the information regarding mortality risk factors comes from small case series, creating statistical instability in the results and concerns about the generalizability of reported findings. In the present large, multi-institutional US cohort, 5 covariates were independent predictors of in-hospital mortality: age in years, epidermal detachment (in percentage of affected BSA), serum bicarbonate level below 20 mmol/L, cancer, and dialysis. This model has similar discrimination to the original SCORTEN (0.816 vs 0.827, P = .72) in the US population. These findings confirm the importance of risk factors such as age, BSA, active/ongoing cancer, and serum bicarbonate level, as identified in SCORTEN, and also highlight dialysis as a significant mortality risk factor to consider.
Dialysis, a proxy for severe renal dysfunction, has been identified previously as an independent risk factor for SJS/TEN-related mortality. Dialysis was associated with a 10.4-fold increased odds of mortality in a cohort of 68 patients, seen from 1984 to 2011, at a burn center in Germany.25 In a Taiwanese population, hemodialysis was associated with 2.5-times increased odds (95% CI, 1.3-4.7) of mortality from SJS/TEN, while a serum creatinine level 1.5 times that of baseline or urine production of less than 0.5 mL/kg over 6 hours were associated with a 7.0-fold increased risk of death (95% CI, 3.5-13.7).20 A retrospective cohort study in France found that patients requiring dialysis while hospitalized with SJS/TEN had higher in-hospital mortality than those not requiring dialysis (81.8% vs 8.8%, P < .001).26 In the present cohort, those undergoing dialysis prior to admission had a more than 15-fold increased risk of death compared with those not undergoing dialysis (OR, 15.94; 95% CI, 3.38-66.30), again confirming the importance of severe renal dysfunction as an SJS/TEN mortality risk factor.
Despite the use of a large, multi-institutional, and geographically diverse cohort, this study has several limitations. First, all participating study sites are tertiary care medical centers, and therefore some patients may have been treated initially at local hospitals prior to being transferred. Prolonged delay in presentation to a tertiary care center may be expected to increase mortality risk27 and also may result in missing or nonuniform data, specifically with regard to BSA, vital signs, and laboratory values. One study found that delay in admission did not diminish the prognostic value of SCORTEN when measured over a period of 5 days,11 suggesting that potential referral delay may not have substantively affected the results observed in this cohort.
Second, there may be heterogeneity in the assessment of epidermal detachment, based on differences between individual physicians, because the data were collected as a part of standard clinical care. Third, as an observational study, the treatment protocol in this cohort was not standardized. While differences in treatment by physicians may lead to variation in mortality rates, no evidence-based standard for best available treatment of SJS/TEN currently exists. Other described SJS/TEN cohorts share these same limitations.5,14-18,22-24 This updated model was developed in a multi-institutional US cohort of patients, and future research should include validation in an external cohort to determine the generalizability of these findings. Finally, because all patients were cared for by inpatient consultative dermatologists at tertiary care centers, the findings in this cohort may not be generalizable to clinical settings in which access to expert dermatologic care, intensive supportive care, or both is not available.
In conclusion, ABCD-10 is a mortality-prediction model designed to predict in-hospital mortality from SJS/TEN. Predicted mortality was calculated by taking the sum of 1 point each for age greater than or equal to 50 years, epidermal detachment greater than 10% of BSA, and serum bicarbonate level below 20 mmol/L; 2 points for the presence of active cancer; and 3 points for dialysis prior to admission. This model accurately predicted mortality within the cohort, with discrimination not significantly different from SCORTEN. Future use of ABCD-10, an updated SJS/TEN mortality risk–prediction model, may provide improved prognostic information for contemporary patients, including those entered in observation studies and therapeutic trials. Additional research is needed to validate the model and better understand the generalizability of these findings.
Corresponding Author: Megan H. Noe MD, MPH, MSCE, University of Pennsylvania, Department of Dermatology, 3400 Civic Center Boulevard–Perelman Center for Advanced Medicine, 7th Floor, South Tower, Philadelphia, PA 19104 (email@example.com).
Accepted for Publication: December 6, 2018.
Published Online: March 6, 2019. doi:10.1001/jamadermatol.2018.5605
Correction: This article was corrected on July 31, 2019, to fix all Hosmer-Lemeshow goodness of fit P value reports to read P = .30 and on September 11, 2019, to correct the spelling of author name Daniela Kroshinsky, MD, MPH.
Author Contributions: Drs Noe and Micheletti had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Rosenbach, Dominguez, Fox, Kroshinsky, Shinkai, Micheletti.
Acquisition, analysis, or interpretation of data: Noe, Rosenbach, Hubbard, Mostaghimi, Cardones, Chen, Cotliar, Davis, Hughey, Kaffenberger, Kroshinsky, Kwong, Miller, Musiek, Ortega-Loayza, Sharon, Shinkai, Summers, Wanat, Wetter, Worswick, Margolis, Gelfand, Micheletti.
Drafting of the manuscript: Noe.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Noe, Hubbard, Margolis.
Administrative, technical, or material support: Davis, Kroshinsky, Wanat, Micheletti.
Study supervision: Rosenbach, Mostaghimi, Ortega-Loayza, Sharon, Gelfand, Micheletti.
Conflict of Interest Disclosures: Dr Rosenbach reports serving as a consultant for Merck, aTyr, and Processa Pharmaceuticals and serving as principal investigator for a trial supported by Processa with funds going to the University of Pennsylvania, all outside the scope of the present work. No other disclosures are reported. Dr Mostaghimi received personal fees from Pfizer and Hims Inc outside the scope of the present work. Dr Musiek reported other conflicts from Soligenix, miRagen, Actilion, Helsinn, Pfizer, and Kyowa and personal fees from Helsinn and Kyowa outside the scope of the present work; she has also served as sub-investigator/technician on several studies in oncology and neurology where she provided the service of biopsies or skin evaluations and was reimbursed for those services. Dr Gelfand served as a consultant for BMS, Boehringer Ingelheim, GSK, Janssen Biologics, Novartis Corp, UCB (DSMB), Sanofi, and Pfizer Inc, receiving honoraria; he receives research grants (to the Trustees of the University of Pennsylvania) from Abbvie, Janssen, Novartis Corp, Celgene, Ortho Dermatologics, and Pfizer Inc; and he received payment for continuing medical education work related to psoriasis that was supported indirectly by Lilly, Ortho Dermatologic, and Novartis. Dr Gelfand is also a co–patent holder of resiquimod for treatment of cutaneous T-cell lymphoma. Dr Gelfand is also a Deputy Editor for the Journal of Investigative Dermatology, receiving honoraria from the Society for Investigative Dermatology. No other disclosures are reported.
Funding/Support: This study was supported in part by National Institutes of Health (NIH Training Grant T32-GM075766 (Dr Noe).
Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Disclaimer: Dr Rosenbach is Deputy Editor of JAMA Dermatology; Dr Mostaghimi is Associate Editor of JAMA Dermatology; and Dr Shinkai is Editor of JAMA Dermatology; however, they were not involved in any of the decisions regarding review of the manuscript or its acceptance.
Additional Contributions: We would like to thank all members of the Society for Dermatology Hospitalists who contributed patients to the cohort.
Create a personal account or sign in to: