The healthiest quartile is not depicted. Trial durations that led to no more than a 1-day difference in mean survival compared with aggressive care, and no more than a 3-day difference in mean survival, were calculated.
eTable 1. Model parameters
eTable 2. Internal and external validation
eFigure 1. State-transition diagram
eFigure 2. Face validation. Predicted survival by SOFA quartile
eFigure 3. Internal and external validation. Modeled vs. predicted survival for validation cohorts
Customize your JAMA Network experience by selecting one or more topics from the list below.
Shrime MG, Ferket BS, Scott DJ, et al. Time-Limited Trials of Intensive Care for Critically Ill Patients With Cancer: How Long Is Long Enough? JAMA Oncol. 2016;2(1):76–83. doi:10.1001/jamaoncol.2015.3336
Time-limited trials of intensive care are commonly used in patients perceived to have a poor prognosis. The optimal duration of such trials is unknown. Factors such as a cancer diagnosis are associated with clinician pessimism and may affect the decision to limit care independent of a patient’s severity of illness.
To identify the optimal duration of intensive care for short-term mortality in critically ill patients with cancer.
Design, Setting, and Participants
Decision analysis using a state-transition microsimulation model was performed to simulate the hospital course of patients with poor-prognosis primary tumors, metastatic disease, or hematologic malignant neoplasms admitted to medical and surgical intensive care units. Transition probabilities were derived from 920 participants stratified by sequential organ failure assessment (SOFA) scores to identify severity of illness. The model was validated in 3 independent cohorts with 349, 158, and 117 participants from quaternary care academic hospitals. Monte Carlo microsimulation was performed, followed by probabilistic sensitivity analysis. Outcomes were assessed in the overall cohort and in solid tumors alone.
Time-unlimited vs time-limited trials of intensive care.
Main Outcomes and Measures
30-day all-cause mortality and mean survival duration.
The SOFA scores at ICU admission were significantly associated with mortality. A 3-, 8-, or 15-day trial of intensive care resulted in decreased mean 30-day survival vs aggressive care in all but the sickest patients (SOFA score, 5-9: 48.4% [95% CI, 48.0%-48.8%], 60.6% [95% CI, 60.2%-61.1%], and 66.8% [95% CI, 66.4%-67.2%], respectively, vs 74.6% [95% CI, 74.3%-75.0%] with time-unlimited aggressive care; SOFA score, 10-14: 36.2% [95% CI, 35.8%-36.6%], 44.1% [95% CI, 43.6%-44.5%], and 46.1% [95% CI, 45.6%-46.5%], respectively, vs 48.4% [95% CI, 48.0%-48.8%] with aggressive care; SOFA score, ≥15: 5.8% [95% CI, 5.6%-6.0%], 8.1% [95% CI, 7.9%-8.3%], and 8.3% [95% CI, 8.1%-8.6%], respectively, vs 8.8% [95% CI, 8.5%-9.0%] with aggressive care). However, the clinical magnitude of these differences was variable. Trial durations of 8 days in the sickest patients offered mean survival duration that was no more than 1 day different from time-unlimited care, whereas trial durations of 10 to 12 days were required in healthier patients. For the subset of patients with solid tumors, trial durations of 1 to 4 days offered mean survival that was not statistically significantly different from time-unlimited care.
Conclusions and Relevance
Trials of ICU care lasting 1 to 4 days may be sufficient in patients with poor-prognosis solid tumors, whereas patients with hematologic malignant neoplasms or less severe illness seem to benefit from longer trials of intensive care.
In the United States, 1 in 5 deaths occurs in the intensive care unit (ICU).1 Although it is not clear that more aggressive care always results in better outcomes, it is difficult to identify which patients will ultimately benefit from intensive care. Clinicians find it stressful to prognosticate.2 Although some studies have suggested that physicians can offer reasonable predictions of survival,3 others disagree, suggesting that clinicians are wrong half the time in predicting who will survive to hospital discharge.4,5 Substantial variability exists between experienced intensivists in providing prognosis for the same patient.6 Differences in opinion about appropriateness of care for a perceived prognosis contribute to clinician burnout7 and may contribute to doubt felt by patients and surrogates.8 Lack of objective data on prognosis may make shared decision making challenging.
Although up to 15% of all patients admitted to the ICU have a diagnosis of cancer,9 a diagnosis of malignant neoplasm is associated with clinician pessimism. In a simulation study of intensivists, a diagnosis of early-stage lung cancer was associated with a 5.8-fold increase in the odds of a recommendation to limit care.6 In practice, a cancer diagnosis has also been associated with a nearly 6-fold increase in ICU care refusal.10 Whereas patients with cancer have traditionally been thought to have a worse prognosis compared with patients without cancer in the ICU, recent studies suggest that short-term mortality may be equivalent in these 2 groups11,12 and may be better predicted by the severity of the acute illness rather than the prognosis of the underlying cancer.13-15 The decision to limit life-sustaining measures is one of the strongest predictors of mortality even after adjusting for severity of illness,16,17 highlighting the crucial importance of making decisions on the basis of objective evidence.
It is common to offer patients a time-limited trial of ICU care.18 However, there is little evidence to guide the duration of such trials. Patients cannot be ethically randomized to trials of ICU care, and observational studies cannot provide information on the counterfactual—that is, what would have happened to the same patient under a time-limited trial of ICU care vs time-unlimited ICU care. Decision analytic models are particularly suited for this question and have been used to establish standards of care in other areas of medicine.19,20
Using data derived from patients with cancer admitted to all ICUs at a quaternary care center, we simulated the hospital course of these patients after admission to the ICU in order to determine the duration of a time-limited trial of ICU care that provides short-term survival comparable to time-unlimited care. The model was subsequently validated in 3 external cohorts of critically ill patients with cancer. We tested the primary hypothesis that an “optimal” duration of a time-limited trial of intensive care for short-term survival can be identified. Although pessimism in the ICU is seen in both nonhematologic and hematologic malignant neoplasms, recent data suggest that patients with the latter have improved survival.12 Therefore, we further tested the hypothesis that “optimal” trial durations are shorter in patients with solid tumors, potentially due to their worse prognosis.
Perceived poor prognosis of critically ill patients with cancer often leads physicians to recommend time-limited “trials” of intensive care in this population, but the optimal duration of intensive care is unknown.
The goal of this study was to determine whether time-limited “trials” of aggressive ICU care in critically ill patients with cancer could provide equivalent survival to time-unlimited care—and if so, to determine the optimal length of such trials.
A probabilistic, state-transition, decision-analytic model was built, using data from a cohort of critically ill patients with cancer (n = 920) admitted between 2001 and 2007 to a quaternary hospital in Boston, and validated on 3 external cohorts of patients admitted between 2002 and 2015 to quaternary hospitals in Boston and Riyadh, Saudi Arabia (n = 349, 158, and 117).
Trial durations of 1 to 4 days appeared sufficient in patients with poor-prognosis solid tumors, whereas patients with hematologic malignant neoplasms or with less severe illness required trials of up to 2 weeks.
In patients with poor-prognosis solid tumors, short, time-limited trials of ICU treatment may be sufficient, but in all other critically ill patients with cancer, longer ICU trials are necessary to achieve optimal survival benefit.
This study was approved by the institutional review boards of each institution. Informed consent was waived due to the retrospective nature of the study.
Data were obtained from a cohort of patients enrolled in the Multiparameter Intelligent Monitoring in Intensive Care II (MIMIC-II) database,21 an automated electronic-capture database of more than 40 000 patients admitted between 2001 and 2007 to all medical and surgical ICUs at the Beth Israel–Deaconess Medical Center (BIDMC) in Boston, Massachusetts. Eligible patients had a diagnosis of metastatic disease, advanced hematologic malignant neoplasms, or a nonmetastatic primary tumor with a poor prognosis (mesothelioma, glioblastoma, pancreatic cancer, small-cell lung cancer, advanced melanoma, or esophageal cancer22-26). The analysis was performed in the overall cohort and restricted only to patients with solid tumors. A total of 920 eligible patients were identified on the basis of attending-level physician medical record review to verify stage and type of cancer, as well as to ascertain baseline patient characteristics.
To measure severity of illness, we used sequential organ failure assessment (SOFA) scores, given their prognostic role in critically ill patients with cancer.15,27,28 The degree of organ dysfunction is scored within the respiratory, coagulation, hepatic, cardiovascular, renal, and central nervous systems to arrive at a composite score, which varies over time with clinical improvement or deterioration.29
We constructed a state-transition microsimulation model to simulate the clinical problem.30 The 5 mutually exclusive states that any patient could enter were aggressive care in the ICU, comfort measures only (CMO) care, inpatient (non-ICU) care, home, and dead (eFigure 1 in the Supplement).
State-transition probabilities, baseline mortality rates, cutoffs for ICU discharge, and ICU mortality rates were calculated from patients in the MIMIC-II database using Kaplan-Meier time-to-event analysis, conditional on individual SOFA scores (eTable 1 in the Supplement). This was done such that the probability of dying, of improving, of deteriorating, or of discharge from ICU to an inpatient floor bed differed on the basis of the SOFA score. When data were missing from the MIMIC-II database, probabilities were extrapolated from patients within that state with neighboring SOFA scores. For patients remaining in the model past the first 3 days, the sample for these calculations became small. As such, transition probabilities from the first day were used. This assumption has the potential of underestimating or overestimating survival for patients; therefore, sensitivity analyses were conducted.
Thirty-one treatment strategies were considered: time-unlimited aggressive care, CMO, and 29 time-varying trial of ICU care strategies (from 1 to 29 days). Outcomes were assessed by SOFA score quartiles. Although the lowest SOFA score quartile (SOFA score, 0-4) was used in model construction to allow the model to track patients as they improved, outcomes were not assessed in this group because a time-limited trial would never be performed in these healthier patients in clinical practice. Under the time-unlimited aggressive care treatment strategy, patients stayed in the aggressive care state until they were either discharged from the ICU or died. Under the CMO treatment strategy, patients stayed in the CMO state until they were either discharged from the ICU or died. Under any of the 29 trial of ICU care strategies, patients began in the aggressive care state and remained until they were discharged from the ICU, died, or until a predetermined number of days had elapsed, at which point they transitioned to the CMO state.
Monte Carlo first- and second-order microsimulations were performed; the model adopted a daily cycle length and followed 10 000 patients per SOFA score through each of the 31 treatment strategies. Because of the short time horizon, discounting was not applied.
Outcomes measured were 30-day survival and mean all-cause survival. The probability of survival to 30 days, the mean survival length, and respective 95% confidence intervals were derived from the aforementioned cohorts. To ask the question, “How long does a trial have to be such that any longer does not increase the chance that I will be alive at 30 days?,” we compared the probability of 30-day survival for time-unlimited aggressive care and 3-, 8-, and 15-day trials of ICU care. These trial durations were chosen on the basis of what is observed in clinical practice.31 To ask the question, “How long does a trial have to be such that my mean survival time is not different (by more than 1 day) from that under time-unlimited intensive care?,” we calculated mean all-cause survival duration and compared this outcome under different strategies. Tests for statistical significance were corrected for multiple comparisons. The model is further described in the eAppendix in the Supplement.
Internal validation was performed on both the aggressive care and CMO treatment strategies. For the former, observed Kaplan-Meier 30-day overall survival curves were constructed. Predicted Kaplan-Meier survival curves were constructed for an equally sized simulated cohort of patients with the same starting SOFA scores as the actual cohort. Predicted and actual survival were compared, and P-for-difference values were calculated using log-rank and Peto and Peto Fleming-Harrington assumptions.32 The CMO model strategy was validated by comparing the mean simulated survival of patients in CMO care with that reported in the literature.
External validation was performed by studying critically ill patients with cancer meeting our inclusion criterion in 3 external cohorts: 349 patients admitted to BIDMC between 2008 and 2012 (MIMIC III), 158 patients admitted to Brigham and Women’s Hospital between 2007 and 2015, and 117 patients admitted to King Abdulaziz Medical City between 2002 and 2013. These represent 2 quaternary academic hospitals in Boston and 1 in Riyadh, Saudi Arabia, respectively. Sample size calculations showed that these validation cohorts had 80% power to detect a survival difference of 15.0%, 18.6%, and 15.5%, respectively. A simulated cohort the size of each external data set was run through the model, adjusting only for SOFA score at the time of admission. The model was explicitly not recalibrated to these data sets. Predicted survival for each cohort was compared against observed.
A number of sensitivity analyses were performed. Under the base-case scenario, a patient undergoing a trial of ICU care was automatically transitioned to CMO at the conclusion of the trial. In clinical practice, this transition would occur only if the patient’s condition did not improve. Therefore, a sensitivity analysis was performed in which a patient remained in the ICU even after the trial ended, as long as there was an improvement in SOFA scores. A second sensitivity analysis was performed on the definition of “equivalent” mean survival. The threshold at which a trial was deemed to offer equivalent mean survival was relaxed from a 1-day to a 3-day difference. We conducted further sensitivity analyses around the clinical trajectory of a patient with a prolonged ICU stay, as described in the eAppendix in the Supplement. Finally, parameter uncertainty was modeled in probabilistic sensitivity analyses.
Data analyses were performed using R, version 3.0 (https://www.r-project.org/), and TreeAge 2013.
A total of 920 patients were identified for the derivation cohort (Table 1). Of these, 69.7% had solid tumors, and the most common malignant neoplasms were lung (16.8%), lymphoma (13.9%), leukemia (9.8%), and pancreas (9.2%). Seventeen percent first received a diagnosis of cancer during their ICU stay; the largest proportion was those with a diagnosis of primary lung cancer (12.8%), followed by lymphoma and pancreatic cancer (both 7.3%). Overall 30-day mortality was 31.1%. Of the patients who died in the first 30 days after ICU admission, 65.7% died in the ICU. Median (IQR) survival was 37 (6-177) days. There were differences between the derivation and validation cohort in terms of severity of illness and case mix.
Face validation of the model is shown in eFigure 2 in the Supplement. For internal and external validation, predicted survival was compared against actual survival in the derivation data set as an internal validation, and against each of the 3 external validation cohorts (eFigure 3 in the Supplement). Model predictions of survival were nearly identical to observed survival for all comers, with no statistically significant differences noted (eTable 2 in the Supplement), despite differences in baseline characteristics. The model predicted that 1.5% of patients in CMO care survived to 30 days or ICU discharge, which is consistent with the literature.33
When compared with aggressive care, time-limited trials of any duration had lower survival probability than aggressive care (Table 2). However, the magnitude of these differences varied. One exception was in the sickest patients, defined as those with SOFA scores of 15 or greater, in which there was no statistical difference between a 15-day trial and time-unlimited aggressive care (30-day survival, 8.3% vs 8.8%; P = .12). In these patients, 30-day survival under an 8-day trial was 8.1% and under a 3-day trial was 5.8% (8-day vs aggressive care, P < .001; 3-day vs aggressive care, P < .001). Three-day trials afforded mean survival that was always statistically significantly lower than aggressive care, although the clinical magnitude of these differences varied.
Under the base-case scenario, patients with SOFA scores between 5 and 14 had an optimal trial duration of approximately 10 to 12 days (Figure, A). For the sickest patients (SOFA score, ≥15), an 8-day trial ensured that mean survival time was no more than 1 day different from that with time-unlimited care.
When the sample was limited to patients with solid tumors, similar patterns were noted for 30-day survival, although survival probabilities were lower under all strategies (Table 2). Three- and 8-day day trials gave lower probabilities of survival when compared with aggressive care for patients in all SOFA quartiles, although similarly the magnitude of these differences varied. For patients with SOFA scores of 10 or more, 15-day trials afforded equivalent survival probabilities compared with aggressive care (SOFA score, 10-14: 36.2% vs 36.6%, P = .13; SOFA score ≥15: 1.5% vs 1.6%, P = .14).
Optimal trial durations for the outcome of mean survival duration were significantly lower in patients with solid tumors, ranging from as long as 4 days (SOFA score, 5-12) to as short as 1 day (SOFA score, 15-20) (Figure, B).
In a sensitivity analysis in which a patient was allowed to continue receiving intensive care after the conclusion of a time-limited trial as long as the patient’s SOFA score was improving daily, optimal trial durations in the sickest patients decreased from 8 to 6 days because the criterion for staying in ICU care was more lenient.
For the mean survival outcome, we used a less rigorous definition by which trials were considered equivalent to aggressive care, using 3-day differences in mean survival to define equivalence. The optimal trial duration predictably decreased for all SOFA scores (Figure, A) in the total cohort. In the solid-tumor subgroup, no difference in the optimal trial duration was found whether a threshold of 1 or 3 days was used to define a difference in mean survival (Figure, B).
To our knowledge, this is the first study offering empirical data on comparative short-term survival outcomes in critically ill patients with cancer under different time-limited vs time-unlimited strategies for intensive care. To date, there has been little research on how to identify the duration of such trials. Using data derived from a large cohort of critically ill patients with cancer, we were able to simulate the clinical course of these patients under different treatment strategies, demonstrate the validity of our model in 3 external cohorts, and perform a number of sensitivity analyses to confirm the robustness of our findings. We show that, in a cohort of critically ill patients with cancer, those with lower severity of acute illness appear to benefit the most from longer trials of intensive care. Whereas 3-day trials always lead to lower short-term survival compared with aggressive care, suggesting that these patients can benefit from longer intensive care, patients, their surrogates, and their clinicians may view the incremental survival benefit from longer trials as marginal and therefore not worthwhile. Cancer patients with solid tumors have a lower survival probability under all treatment strategies. Trial durations necessary to offer equivalent mean survival duration in these patients are as short as 1 to 4 days.
Our study is novel. The current literature on this topic is limited to either expert opinion18,34 or to descriptive studies on what is commonly done in clinical practice.31 One observational trial does support a trial duration of longer than 3 days in critically ill patients with cancer, on the basis of the observation that nonbedridden patients with cancer have overall improved hospital survival if they are given the opportunity to receive intensive care for at least 4 days.14 However, a direct comparison between different trial durations could not be made. A randomized clinical trial will never be performed on this issue because it would be unethical, leading us to approach this problem with decision analytic models.
A major strength of this study is the rigor with which we developed and tested our model. Because of the unique nature of the MIMIC-II database, every single admission to all ICUs at BIDMC was captured between 2001 and 2007, ensuring no selection bias. Patients identified as having cancer had their discharge summaries manually reviewed by attending-level intensivists and oncologists. The model was validated in 3 external cohorts despite differences in case mix, demonstrating the clinical utility of the SOFA score in predicting survival. We believe that the results of our study have useful clinical implications because the information provided is objective and can be used in family meetings to help set expectations for a reasonable trial duration. To provide more information for these discussions, we specifically addressed the question of optimal trial duration in 2 different ways: by looking at 30-day survival probability and by looking at mean survival duration. Whereas a 1-day mean survival difference may not seem clinically meaningful to clinicians, for some patients with a limited life expectancy, even a small gain in benefit at a huge cost has been deemed meaningful,35 although for others, quality rather than length of life is more important in decision making.36
This study has some limitations. Although our outcome was short-term mortality, functional status and quality of life affect decision making. However, mortality remains an important outcome, independent of these factors; one study in patients who have experienced critical illness indicated that the majority would be willing to undergo intensive care even for brief periods of life prolongation regardless of quality of life or functional status.37 We focused our study on patients with cancer because time-limited trials are often reserved for patients who are perceived (correctly or not) to have limited life expectancy or poor quality of life. A cancer diagnosis is strongly associated with pessimism and/or the decision to limit care in the ICU,10 even in the case of early-stage cancer.6 Nononcologists may be unaware that even in the extreme case of patients with brain metastases, mean survival is typically not on the order of weeks but months to years, with 6.9% alive at 2 years.38 We believe that our novel approach to investigating time-limited trials will provide a framework applicable to other patient populations in which pessimism exists, such as those with chronic respiratory disease39 or extremely old patients.40
We demonstrate that in critically ill patients with cancer, those with hematologic malignant neoplasms or less severe illness seem to benefit the most from longer trials of intensive care. For patients with poor-prognosis solid tumors, shorter trial durations of 1 to 4 days may be sufficient. These results, in combination with individual preference, may be helpful in guiding decision making at the end of life in this patient population.
Accepted for Publication: July 20, 2015.
Corresponding Author: Mark G. Shrime, MD, MPH, PhD, Emergency Department, Massachusetts Eye and Ear Infirmary, 243 Charles St, Boston, MA 02114 (firstname.lastname@example.org).
Published Online: October 15, 2015. doi:10.1001/jamaoncol.2015.3336.
Author Contributions: Drs Shrime and Lai had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Shrime, Arabi, Hunink, Celi, Lai.
Acquisition, analysis, or interpretation of data: Shrime, Ferket, Scott, Lee, Barragan-Bradford, Pollard, Al-Dorzi, Baron, Hunink, Celi, Lai.
Drafting of the manuscript: Shrime, Lai.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Shrime, Lai.
Obtained funding: Shrime.
Administrative, technical, or material support: Lee, Barragan-Bradford, Pollard, Baron, Celi, Lai.
Study supervision: Arabi, Hunink, Lai.
Conflict of Interest Disclosures: Dr Shrime received speaking fees from Ethicon in January 2014 for a talk unrelated to this article. Dr Hunink has received personal fees from Cambridge University Press, grants and nonfinancial support from the European Society of Radiology, and nonfinancial support from the European Institute for Biomedical Imaging Research, all outside the present work. No other disclosures are reported.
Funding/Support: This study received partial support from National Institutes of Health grants NIH R01 HL112747 to Dr Baron for collection of data, NIH-NIEHS K23 ES023700 to Dr Lai as part of a career development award, and NIH-NIBIB R01 EB017205-01A1 for support of the MIMIC database.
Role of Funder/Sponsor: The National Institutes of Health had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Previous Presentations: This study was previously presented in part on May 22, 2012, at the American Thoracic Society Annual Meeting in San Francisco, California; and on October 20, 2013, at the Society for Medical Decision Making Annual Meeting in Baltimore, Maryland.
Additional Contributions: We would like to acknowledge the BWH Registry of Critical Illness for providing data for the validation cohorts, as well as Alistair Johnson, DPhil, Massachusetts Institute of Technology, Harvard-MIT Division of Health Sciences and Technology, Boston, Massachusetts, for his help in data reconciliation for the MIMIC III database. Dr Johnson did not receive compensation for his contribution.