Laboratory Findings Associated With Severe Illness and Mortality Among Hospitalized Individuals With Coronavirus Disease 2019 in Eastern Massachusetts | Critical Care Medicine | JAMA Network Open | JAMA Network
[Skip to Navigation]
Sign In
Figure 1.  Model Performance in Test Set
Model Performance in Test Set

A, Coronavirus Disease 2019 (COVID-19)–associated severe outcome. B, COVID-19–associated mortality. AUC indicates area under the curve.

Figure 2.  Quintile Plots of Outcomes in Independent Testing Cohort
Quintile Plots of Outcomes in Independent Testing Cohort
Figure 3.  Kaplan-Meier Curves in Independent Testing Cohort
Kaplan-Meier Curves in Independent Testing Cohort
Table 1.  Sociodemographic Characteristics of Training and Test Sets
Sociodemographic Characteristics of Training and Test Sets
Table 2.  Model Coefficients
Model Coefficients
1.
Guan  W-J, Ni  Z-Y, Hu  Y,  et al; China Medical Treatment Expert Group for Covid-19.  Clinical characteristics of coronavirus disease 2019 in China.   N Engl J Med. 2020;382(18):1708-1720. doi:10.1056/NEJMoa2002032PubMedGoogle ScholarCrossref
2.
Grasselli  G, Pesenti  A, Cecconi  M.  Critical care utilization for the COVID-19 outbreak in Lombardy, Italy: early experience and forecast during an emergency response.   JAMA. 2020;323(16):1545-1546. Published online March 13, 2020. doi:10.1001/jama.2020.4031PubMedGoogle ScholarCrossref
3.
Bhatraju  PK, Ghassemieh  BJ, Nichols  M,  et al.  Covid-19 in critically ill patients in the Seattle region—case series.   N Engl J Med. 2020;382(21):2012-2022. doi:10.1056/NEJMoa2004500PubMedGoogle ScholarCrossref
4.
Richardson  S, Hirsch  JS, Narasimhan  M,  et al; the Northwell COVID-19 Research Consortium.  Presenting characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the New York City area.   JAMA. 2020;323(20):2052-2059. doi:10.1001/jama.2020.6775PubMedGoogle ScholarCrossref
5.
Ruan  Q, Yang  K, Wang  W, Jiang  L, Song  J.  Clinical predictors of mortality due to COVID-19 based on an analysis of data of 150 patients from Wuhan, China.   Intensive Care Med. 2020;46(5):846-848. doi:10.1007/s00134-020-05991-xPubMedGoogle ScholarCrossref
6.
Zhou  F, Yu  T, Du  R,  et al.  Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study.   Lancet. 2020;395(10229):1054-1062. doi:10.1016/S0140-6736(20)30566-3 PubMedGoogle ScholarCrossref
7.
Du  Y, Tu  L, Zhu  P,  et al.  Clinical features of 85 fatal cases of COVID-19 from Wuhan: a retrospective observational study.   Am J Respir Crit Care Med. 2020;201(11):1372-1379. doi:10.1164/rccm.202003-0543OCPubMedGoogle ScholarCrossref
8.
Henry  BM, de Oliveira  MHS, Benoit  S, Plebani  M, Lippi  G.  Hematologic, biochemical and immune biomarker abnormalities associated with severe illness and mortality in coronavirus disease 2019 (COVID-19): a meta-analysis.   Clin Chem Lab Med. 2020;58(7):1021-1028. doi:10.1515/cclm-2020-0369 PubMedGoogle ScholarCrossref
9.
Consortium for Clinical Characterization of Covid19 by EHR (4CE). i2b2 tranSMART Foundation. Published April 5, 2020. Accessed April 19, 2020. https://transmartfoundation.org/covid-19-community-project/
10.
Pencina  MJ, Goldstein  BA, D’Agostino  RB.  Prediction models—development, evaluation, and clinical application.   N Engl J Med. 2020;382(17):1583-1586. doi:10.1056/NEJMp2000589 PubMedGoogle ScholarCrossref
11.
Nalichowski  R, Keogh  D, Chueh  HC, Murphy  SN.  Calculating the benefits of a Research Patient Data Repository.   AMIA Annu Symp Proc. 2006;2006:1044.PubMedGoogle Scholar
12.
Murphy  SN, Weber  G, Mendis  M,  et al.  Serving the enterprise and beyond with informatics for integrating biology and the bedside (i2b2).   J Am Med Inform Assoc. 2010;17(2):124-130. doi:10.1136/jamia.2009.000893 PubMedGoogle ScholarCrossref
13.
Charlson  M, Szatrowski  TP, Peterson  J, Gold  J.  Validation of a combined comorbidity index.   J Clin Epidemiol. 1994;47(11):1245-1251. doi:10.1016/0895-4356(94)90129-5 PubMedGoogle ScholarCrossref
14.
Healthcare Cost and Utilization Project (HCUP). HCUP-US Tools & Software Page. Clinical Classifications Software (CCS) for ICD-9-CM. Accessed September 6, 2020. https://www.hcup-us.ahrq.gov/tools_software.jsp
15.
Tibshirani  R.  Regression shrinkage and selection via the lasso.   J R Stat Soc Ser B Methodol. 1996;58(1):267-288. doi:10.1111/j.2517-6161.1996.tb02080.x Google Scholar
16.
The R Project for Statistical Computing. The R Foundation. Accessed September 6, 2020. http://www.R-project.org
17.
Pei  G, Zhang  Z, Peng  J,  et al.  Renal involvement and early prognosis in patients with COVID-19 pneumonia.   J Am Soc Nephrol. 2020;31(6):1157-1165. doi:10.1681/ASN.2020030276PubMedGoogle ScholarCrossref
18.
Wynants  L, Van Calster  B, Collins  GS,  et al.  Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal.   BMJ. 2020;369:m1328. doi:10.1136/bmj.m1328 PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Original Investigation
    Health Informatics
    October 30, 2020

    Laboratory Findings Associated With Severe Illness and Mortality Among Hospitalized Individuals With Coronavirus Disease 2019 in Eastern Massachusetts

    Author Affiliations
    • 1Center for Quantitative Health, Division of Clinical Research, Massachusetts General Hospital, Boston, Massachusetts
    • 2Research Information Science and Computing, Mass General Brigham, Somerville, Massachusetts
    JAMA Netw Open. 2020;3(10):e2023934. doi:10.1001/jamanetworkopen.2020.23934
    Key Points

    Question  How well can sociodemographic features, laboratory values, and comorbidities of individuals hospitalized with coronavirus disease 2019 (COVID-19) in Eastern Massachusetts through June 5, 2020, predict a severe illness course?

    Findings  In this cohort study of 2511 hospitalized individuals positive for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) by polymerase chain reaction who were admitted to 1 of 6 hospitals, 215 (8.6%) were admitted to the instensive care unit, 164 (6.5%) required mechanical ventilation, and 292 (11.6%) died. In a risk prediction model, 212 deaths (78%) occurred in the top mortality-risk quintile.

    Meaning  Simple prediction models may assist in risk stratification among hospitalized patients with COVID-19.

    Abstract

    Importance  The coronavirus disease 2019 (COVID-19) pandemic has placed unprecedented stress on health systems across the world, and reliable estimates of risk for adverse hospital outcomes are needed.

    Objective  To quantify admission laboratory and comorbidity features associated with critical illness and mortality risk across 6 Eastern Massachusetts hospitals.

    Design, Setting, and Participants  Retrospective cohort study of all individuals admitted to the hospital who tested positive for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) by polymerase chain reaction across these 6 hospitals through June 5, 2020, using hospital course, prior diagnoses, and laboratory values in emergency department and inpatient settings from 2 academic medical centers and 4 community hospitals. The data were extracted on June 11, 2020, and the analysis was conducted from June to July 2020.

    Exposures  SARS-CoV-2.

    Main Outcomes and Measures  Severe illness defined by admission to intensive care unit, mechanical ventilation, or death.

    Results  Of 2511 hospitalized individuals who tested positive for SARS-CoV-2 (of whom 50.9% were male, 53.9% White, and 27.0% Hispanic, with a mean [SD ]age of 62.6 [19.0] years), 215 (8.6%) were admitted to the intensive care unit, 164 (6.5%) required mechanical ventilation, and 292 (11.6%) died. L1-regression models developed in 3 of these hospitals yielded an area under the receiver operating characteristic curve of 0.807 for severe illness and 0.847 for mortality in the 3 held-out hospitals. In total, 212 of 292 deaths (72.6%) occurred in the highest-risk mortality quintile.

    Conclusions and Relevance  In this cohort, specific admission laboratory studies in concert with sociodemographic features and prior diagnosis facilitated risk stratification among individuals hospitalized for COVID-19.

    Introduction

    With the rapid spread of coronavirus disease 2019 (COVID-19), efforts to predict clinical outcomes and stratify risk have taken on greater urgency as a means of allocating resources and targeting interventions. A recent report of 1099 admitted individuals from China found that 5.0% required intensive care unit (ICU) transfer, and 2.3% required mechanical ventilation.1 In Lombardy, Italy, approximately 16% of test-positive individuals required ICU admission.2 In the United States, the characteristics of admitted patients may differ somewhat. A recent case series from the Seattle area in Washington described 24 ICU-admitted patients, of whom 75% required mechanical ventilation.3 In one of the largest US studies, to date, among a series of 2634 hospitalized patients in New York, New York, who died or were discharged from the hospital, 12.2% had required mechanical ventilation.4

    Given the constrained resources for treatment of COVID-19, particularly with regard to mechanical ventilation, simple approaches to stratifying morbidity and mortality risk at time of hospitalization are needed. In cohorts ranging from 100 to 200 patients, multiple laboratory studies have been associated with mortality risk, including elevated ferritin, troponin, and C-reactive peptide levels,5 elevated D-dimer level,6 and low eosinophil count.7 Most recently, a meta-analysis including a total of 3377 patients identified multiple blood cell indices as most strongly predictive of mortality.8

    Electronic health records may facilitate a rapid and efficient investigation of clinical cohorts and may form the basis of efforts by consortia to address COVID-19 at scale.9 Here, we examined records from 2 academic medical centers and 4 affiliated community hospitals in Eastern Massachusetts. We applied data from 3 of these hospitals to generate simple and transparent models to estimate the risk of a severe hospital course, characterized by the need for mechanical ventilation, ICU level of care, or risk for death, and we validated these results in 3 held-out hospitals, including another academic medical center and 2 community hospitals, as a starting point for generalizable efforts at clinical risk stratification.10 We hypothesized that clinical and laboratory data available at data collection could efficiently stratify risk for a more severe in-hospital course or death.

    Methods
    Patients

    The full cohort included all individuals aged 18 of age or older hospitalized at any of the 2 academic medical centers and 4 community affiliate hospitals between March 1 and June 5, 2020, with a documented polymerase chain reaction–positive test result for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) within 5 days of admission. We excluded patients with a severe outcome on the same day of admission (for whom a prediction based on laboratory studies would be uninformative) and patients transferred from outside hospitals. For all of these individuals, data on prior diagnosis and course during the admission were extracted from the Partners Research Patient Data Registry11 and the Enterprise Data Warehouse and used to generate an i2b2 datamart.12 Data were augmented with age, sex, race, and race/ethnicity from the same source. The enterprise laboratory feed was used to extract SARS-CoV-2 test order and results, as well as additional laboratory values (eTable 1 in the Supplement). Laboratory values available in at least 80% of individuals were included in subsequent analysis as continuous measures, after Winsorization at the 99th percentile but otherwise without transformation, along with laboratory-specific high and low flags. As an aggregate measure of comorbidity, the Charlson comorbidity index was calculated using coded International Classification of Diseases, Ninth Revision (ICD-9) and International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) diagnostic codes obtained from the electronic health record as previously described.13

    The study protocol was approved by the Partners HealthCare Human Research Committee. No participant contact was required in this study, which relied on secondary use of data produced by routine clinical care, allowing for the waiver of a requirement for informed consent as detailed by 45 CFR 46.116. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline for cohort studies was applied.

    Study Design and Statistical Analysis

    We included all newly hospitalized individuals with a SARS-CoV-2–positive polymerase chain reaction test result within 5 days of admission. Patients were followed up from time of admission to hospital discharge or death, with follow-up censored at time of hospital discharge for consistency with similar publications and because mortality data are not yet available from independent sources (ie, National Death Index or equivalent). The 2 primary outcomes of interest were (1) COVID-19–associated severe illness, including any of the following: admission to the ICU, mechanical ventilation, or mortality, and (2) mortality. The former composite outcome was selected to avoid the problem of competing risk (ie, individuals with severe illness who die before either admission to the ICU or mechanical ventilation). We selected the earliest laboratory values and vital signs associated with the admission, including those measured in the emergency department. Prior ICD diagnoses were grouped using a Healthcare Utilization Project Clinical Classification Software hierarchy.14 The log-transformed counts of each Clinical Classification Software diagnosis group were used as predictors. Beyond descriptive analysis, we report appropriate univariate comparisons (ie, the χ2 test for binary variables and the t test for continuous measures) followed by L1-penalized logistic regression, or the least absolute shrinkage and selection operator (lasso),15 to identify a parsimonious model with sociodemographic features, baseline vital signs, prior diagnosis, and laboratory values as candidate predictors. The hospitals were divided into a training cohort (composed of 1 academic center and 2 community hospital) and an evaluative cohort (composed of the other 1 academic medical center and 2 community hospitals). The lasso was applied to all participants with complete laboratory studies in the training cohort and to the performance of the model evaluated in the wholly separate evaluative cohort. Model fitting used all individuals in the training set, with median imputation of missing data (including laboratory values); for individuals in the testing set, we substituted missing values with median values from the training set. Model performance was characterized using standard metrics of discrimination and calibration, focusing on the 5 quintiles of risk determined in the training data set, without recalibration.

    These logistic regression models offer advantages in interpretability but fail to consider censoring. Therefore, to better characterize model performance in the testing set, for comparison we also used survival analysis, right-censoring at the time of hospital discharge or the end of available data (June 5, 2020), presenting Kaplan-Meier curves comparing risk quintile groups. All analyses used R, version 4.0.0.16 The log-rank test statistic was used to compare quintile curves.

    Results

    The 2511 individuals hospitalized through June 5, 2020, included 1348 (53.7%) at academic medical centers and 1163 (46.3%) at community hospitals; 1277 (50.9%) were male, 1354 (53.9%) were White, and 679 (27.0%) were Hispanic; the mean (SD) age was 62.6 (19.0) years (Table 1). In all, 215 individuals (8.6%) were admitted to the ICU, 164 (6.5%) required mechanical ventilation, and 292 (11.6%) died. Of the 2511 total hospitalizations, 634 occurred in the testing cohort, and 1877 occurred in the training cohort. Laboratory values are summarized in eTable 2 and illustrated in eFigure in the Supplement.

    We used L1-penalized regression to train a prediction model based on admission characteristics, prior diagnosis, and laboratory values in 1 academic medical center and 2 community hospitals (Table 2). For severe illness, notable features included lymphocytopenia (low absolute lymphocyte count model coefficient: 0.3049), eosinopenia (eosinophil count model coefficient: −0.3129) and neutrophilia (high absolute neutrophil count model coefficient: 0.2332) as well as markers of diminished renal function (blood urea nitrogen model coefficient: 0.0091; high creatinine level model coefficient: 0.2546; low glomerular filtration rate model coefficient: 0.0736) (Table 2). For mortality (Table 2), the features were generally similar, with the addition of the presence of nucleated red blood cells (high absolute nucleated red blood cell count model coefficient: 0.3478) and other abnormal red blood cell indices (low mean corpuscular hemoglobin concentration model coefficient: 0.0099); red blood cell distribution width model coefficient: 0.0451), procalcitonin level (model coefficient: 0.1823), and greater representation of prior diagnosed codes consistent with pulmonary disease (prior diagnosis of COPD model coefficient: 0.0454; prior diagnosis of lung cancer model coefficient: 0.0389; prior diagnosis of respiratory failure model coefficient: 0.0373) (eTable 3 and eTable 4 in the Supplement report coefficients for features included in penalize regression, without shrinkage).

    In the independent testing set composed of a second academic medical center and 2 other community hospitals, the COVID-19–associated severe illness model yielded an area under the curve of 0.807 (Figure 1A), with a sensitivity of 60.6% and a specificity of 88.9% at the top risk quintile (the positive predictive value was 54.7%, whereas the negative predictive value was 91.1%). For the mortality model, the area under the curve was 0.847 (Figure 1B), with a sensitivity of 78.0% and a specificity of 87.5% (the positive predictive value was 45.6%, whereas the negative predictive value was 96.7%). Both models exhibited substantial lift, with the highest-risk quintile enriched for adverse outcomes in the test cohort (severe illness highest quintile lift: 3.0; mortality model highest quintile lift: 3.9) (Figure 2A and B).

    For illustrative purposes, we also examined the COVID-19–associated severe illness risk quintile (from the model incorporating all adverse outcomes) and mortality risk quintile in Kaplan-Meier survival curves, with curves censored at the time of hospital discharge, June 5, 2020 (ie, end of available follow-up), or 14 days, whichever came first (Figure 3). Quintiles were significantly associated with predicted outcome by log-rank test (χ2 = 818; P < .001). In total, 212 of 292 deaths (78%) occurred in the highest-risk mortality quintile.

    Discussion

    In this study of 2511 individuals with COVID-19 hospitalized at academic medical centers and community hospitals in Eastern Massachusetts through June 5, 2020, 8.6% were admitted to the ICU, 6.5% required mechanical ventilation, and 11.6% died. In general, abnormal hematologic measures (including neutrophilia, lymphocytopenia, and eosinopenia), as well as diminished renal function, were associated with a greater risk of severe hospital course. Measures of prior pulmonary disease and of red blood cell abnormalities were also represented in the risk for mortality.

    Discrimination of both models appears promising, identifying a high-risk quintile with reasonable sensitivity and specificity. Likewise, survival curves support the informativeness of the high-risk quintile and indicate that the results are not an artifact of differential attrition or shorter hospital stay. Predictions may be most useful during the initial week of hospitalization; a useful next-step study could examine whether rerunning models with additional laboratory studies, or incorporating other biomarkers, can improve longer-term prediction.

    Our results are consistent with a recently reported study associating renal involvement with mortality.17 Multiple smaller cohorts have also reported laboratory features associated with morbidity and mortality among hospitalized patients with COVID-19. For example, a retrospective cohort study from Wuhan, China, of 191 hospitalized patients found that older age and a higher d-dimer level at admission were associated with risk of death.6

    Among 95 fatal cases of COVID-19, low eosinophil count at admission was also common.7 An elevated ferritin level was also associated with mortality in a retrospective cohort study of 120 patients from Wuhan,5 along with elevated troponin and C-reactive peptide levels. Our results also confirm and extend those of the largest meta-analysis, to date, encompassing 3377 patients and implicating hematologic measures as well as renal function,8 in addition to markers of tissue injury more generally.

    In developing these simple prediction models, we were mindful of the recent frameworks for and criticisms of such models,10,18 particularly the recognition that poorly validated or calibrated models may cause more harm than good. Initial models are likely optimistic (ie, overfit to data) and biased (ie, by nonrepresentative samples), with a lack of transparency.18 On the other hand, strategies that allow for risk stratification are particularly necessary in an environment of constrained resources. As such, we report these results in the hope that they will provide simple base-case models for others to improve on. Undoubtedly, the application of more complex models will yield further improvement in model fit, but whether the degree of improvement is sufficient to offset the added complexity of clinical implementation and reduced interpretability will merit careful consideration.

    Limitations

    We note the multiple limitations that likely diminished model performance. First, because these are open hospital systems rather than closed networks, the lack of documented prior diagnoses does not preclude their presence for individuals who may receive care elsewhere. For this reason, we excluded hospital transfers because prior documentation of comorbidity is likely to be biased. However, such missing data are likely to diminish the predictive power of any given diagnosis, such that our model performance estimates are likely to be conservative. In addition, many laboratory values are highly nonnormal, such that incorporation of more specific transformations or cutpoints would likely improve model performance; we elected to incorporate standard high or low flags plus continuous measures, rather than adopting specific transformations for each value, which would risk overfitting or would diminish generalizability but likely extract additional information. Efforts to aggregate laboratory data across international health systems will provide an opportunity to explore such transformations if individual-level data become accessible.9

    Conclusions

    The results of this cohort study suggest the utility of laboratory values in combination with documented comorbidities and sociodemographic features in identifying individuals at particularly high risk for more severe hospital course. Notably, by validating in distinct hospitals (albeit within a single geographic region), our estimates of model performance are likely to be less optimistic but still suggest that generalizability should be good. These admission models also provide an opportunity for comparison as more sophisticated models are developed, particularly those incorporating additional biological measures. To the extent hospital resources are constrained, the ability to target resources to highest-risk individuals is likely to be valuable, and expansion and refinement of risk models may represent a useful approach to optimizing care.

    Back to top
    Article Information

    Accepted for Publication: September 2, 2020.

    Published: October 30, 2020. doi:10.1001/jamanetworkopen.2020.23934

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Castro VM et al. JAMA Network Open.

    Corresponding Author: Roy H. Perlis, MD, MSc, Center for Quantitative Health, Division of Clinical Research, Massachusetts General Hospital, 185 Cambridge St, 6th Floor, Boston, MA 02114 (rperlis@mgh.harvard.edu).

    Author Contributions: Dr Perlis had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: All authors.

    Acquisition, analysis, or interpretation of data: All authors.

    Drafting of the manuscript: All authors.

    Critical revision of the manuscript for important intellectual content: All authors.

    Statistical analysis: Castro, McCoy.

    Administrative, technical, or material support: All authors.

    Supervision: Perlis.

    Conflict of Interest Disclosures: Dr Perlis has received consulting fees from Burrage Capital, Genomind, RID Ventures, and Takeda. He holds equity in Outermost Therapeutics and Psy Therapeutics. Dr McCoy has received research funding from the Stanley Center at the Broad Institute, the Brain and Behavior Research Foundation, National Institute of Mental Health, National Human Genome Research Institute Home, and Telefonica Alfa. No other disclosures were reported.

    Disclaimer: Dr Perlis is an Associate Editor at JAMA Network Open; however, he had no role in the editorial review or decision to accept the manuscript for publication.

    Additional Contributions: The authors would like to thank the Partners Research Patient Data Registry and Analytics Enclave team members for their support in making up-to-date electronic health record data available for this research. No financial compensation was provided specifically for this work.

    References
    1.
    Guan  W-J, Ni  Z-Y, Hu  Y,  et al; China Medical Treatment Expert Group for Covid-19.  Clinical characteristics of coronavirus disease 2019 in China.   N Engl J Med. 2020;382(18):1708-1720. doi:10.1056/NEJMoa2002032PubMedGoogle ScholarCrossref
    2.
    Grasselli  G, Pesenti  A, Cecconi  M.  Critical care utilization for the COVID-19 outbreak in Lombardy, Italy: early experience and forecast during an emergency response.   JAMA. 2020;323(16):1545-1546. Published online March 13, 2020. doi:10.1001/jama.2020.4031PubMedGoogle ScholarCrossref
    3.
    Bhatraju  PK, Ghassemieh  BJ, Nichols  M,  et al.  Covid-19 in critically ill patients in the Seattle region—case series.   N Engl J Med. 2020;382(21):2012-2022. doi:10.1056/NEJMoa2004500PubMedGoogle ScholarCrossref
    4.
    Richardson  S, Hirsch  JS, Narasimhan  M,  et al; the Northwell COVID-19 Research Consortium.  Presenting characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the New York City area.   JAMA. 2020;323(20):2052-2059. doi:10.1001/jama.2020.6775PubMedGoogle ScholarCrossref
    5.
    Ruan  Q, Yang  K, Wang  W, Jiang  L, Song  J.  Clinical predictors of mortality due to COVID-19 based on an analysis of data of 150 patients from Wuhan, China.   Intensive Care Med. 2020;46(5):846-848. doi:10.1007/s00134-020-05991-xPubMedGoogle ScholarCrossref
    6.
    Zhou  F, Yu  T, Du  R,  et al.  Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study.   Lancet. 2020;395(10229):1054-1062. doi:10.1016/S0140-6736(20)30566-3 PubMedGoogle ScholarCrossref
    7.
    Du  Y, Tu  L, Zhu  P,  et al.  Clinical features of 85 fatal cases of COVID-19 from Wuhan: a retrospective observational study.   Am J Respir Crit Care Med. 2020;201(11):1372-1379. doi:10.1164/rccm.202003-0543OCPubMedGoogle ScholarCrossref
    8.
    Henry  BM, de Oliveira  MHS, Benoit  S, Plebani  M, Lippi  G.  Hematologic, biochemical and immune biomarker abnormalities associated with severe illness and mortality in coronavirus disease 2019 (COVID-19): a meta-analysis.   Clin Chem Lab Med. 2020;58(7):1021-1028. doi:10.1515/cclm-2020-0369 PubMedGoogle ScholarCrossref
    9.
    Consortium for Clinical Characterization of Covid19 by EHR (4CE). i2b2 tranSMART Foundation. Published April 5, 2020. Accessed April 19, 2020. https://transmartfoundation.org/covid-19-community-project/
    10.
    Pencina  MJ, Goldstein  BA, D’Agostino  RB.  Prediction models—development, evaluation, and clinical application.   N Engl J Med. 2020;382(17):1583-1586. doi:10.1056/NEJMp2000589 PubMedGoogle ScholarCrossref
    11.
    Nalichowski  R, Keogh  D, Chueh  HC, Murphy  SN.  Calculating the benefits of a Research Patient Data Repository.   AMIA Annu Symp Proc. 2006;2006:1044.PubMedGoogle Scholar
    12.
    Murphy  SN, Weber  G, Mendis  M,  et al.  Serving the enterprise and beyond with informatics for integrating biology and the bedside (i2b2).   J Am Med Inform Assoc. 2010;17(2):124-130. doi:10.1136/jamia.2009.000893 PubMedGoogle ScholarCrossref
    13.
    Charlson  M, Szatrowski  TP, Peterson  J, Gold  J.  Validation of a combined comorbidity index.   J Clin Epidemiol. 1994;47(11):1245-1251. doi:10.1016/0895-4356(94)90129-5 PubMedGoogle ScholarCrossref
    14.
    Healthcare Cost and Utilization Project (HCUP). HCUP-US Tools & Software Page. Clinical Classifications Software (CCS) for ICD-9-CM. Accessed September 6, 2020. https://www.hcup-us.ahrq.gov/tools_software.jsp
    15.
    Tibshirani  R.  Regression shrinkage and selection via the lasso.   J R Stat Soc Ser B Methodol. 1996;58(1):267-288. doi:10.1111/j.2517-6161.1996.tb02080.x Google Scholar
    16.
    The R Project for Statistical Computing. The R Foundation. Accessed September 6, 2020. http://www.R-project.org
    17.
    Pei  G, Zhang  Z, Peng  J,  et al.  Renal involvement and early prognosis in patients with COVID-19 pneumonia.   J Am Soc Nephrol. 2020;31(6):1157-1165. doi:10.1681/ASN.2020030276PubMedGoogle ScholarCrossref
    18.
    Wynants  L, Van Calster  B, Collins  GS,  et al.  Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal.   BMJ. 2020;369:m1328. doi:10.1136/bmj.m1328 PubMedGoogle ScholarCrossref
    ×