CRH indicates chemotherapy-related hospitalization.
Data points represent overlapping strata of predicted risk.
eTable. Beta coefficients from the multivariable regression model
eFigure 1. Receiver operating characteristic curve
eFigure 2. Predicted risk of hospitalization in patients with and without chemotherapy-related hospitalization
Customize your JAMA Network experience by selecting one or more topics from the list below.
Brooks GA, Kansagra AJ, Rao SR, Weitzman JI, Linden EA, Jacobson JO. A Clinical Prediction Model to Assess Risk for Chemotherapy-Related Hospitalization in Patients Initiating Palliative Chemotherapy. JAMA Oncol. 2015;1(4):441–447. doi:https://doi.org/10.1001/jamaoncol.2015.0828
Chemotherapy-related hospitalizations in patients with advanced cancer are common, distressing, and costly. Methods to identify patients at high risk of chemotherapy toxic effects will permit development of targeted strategies to prevent chemotherapy-related hospitalizations.
To demonstrate the feasibility of using readily available clinical data to assess patient-specific risk of chemotherapy-related hospitalization.
Design, Setting, and Participants
Nested case-control study conducted from January 2003 through December 2011 at the Mass General/North Shore Cancer Center, a community-based cancer center in northeastern Massachusetts. The parent cohort included 1579 consecutive patients with advanced solid-tumor cancer receiving palliative-intent chemotherapy. Case patients (n = 146) included all patients from the parent cohort who experienced a chemotherapy-related hospitalization. Controls (n = 292) were randomly selected from 1433 patients who did not experience a chemotherapy-related hospitalization.
Putative risk factors for chemotherapy-related hospitalization—including patient characteristics, treatment characteristics, and pretreatment laboratory values—were abstracted from medical records. Multivariable logistic regression was used to model the patient-specific risk of chemotherapy-related hospitalization.
Main Outcomes and Measures
Chemotherapy-related hospitalization, as adjudicated by the oncology clinical care team within a systematic quality-assessment program.
A total of 146 (9.2%) of 1579 patients from the parent cohort experienced a chemotherapy-related hospitalization. In multivariate regression, 7 variables were significantly associated with chemotherapy-related hospitalization: age, Charlson comorbidity score, creatinine clearance, calcium level, below-normal white blood cell and/or platelet count, polychemotherapy (vs monotherapy), and receipt of camptothecin chemotherapy. The median predicted risk of chemotherapy-related hospitalization was 6.0% (interquartile range [IQR], 3.6%-11.4%) in control patients and 14.7% (IQR, 6.8%-22.5%) in case patients. The bootstrap-adjusted C statistic was 0.71 (95% CI, 0.66-0.75). At a risk threshold of 15%, the model exhibited a sensitivity of 49% (95% CI, 41%-57%) and a specificity of 85% (95% CI, 81%-89%) for predicting chemotherapy-related hospitalization.
Conclusions and Relevance
In patients initiating palliative chemotherapy for cancer, readily available clinical data were associated with the patient-specific risk of chemotherapy-related hospitalization. External validation and evaluation in the context of a clinical decision support tool are warranted.
In patients with advanced cancer, hospitalization is a common and costly adverse event.1-3 While most hospitalizations in this patient population are precipitated by cancer-related symptoms, approximately 30% are triggered by adverse effects of chemotherapy.4-6 Hospitalizations related to chemotherapy adverse effects are a plainly undesirable outcome, particularly when the goals of chemotherapy are palliative.
When making chemotherapy treatment plans, oncologists use many indicators to identify patients at risk for adverse effects. Performance status, a clinical estimate of functional status, is the most important of these indicators, and patients with poor performance status (eg, an Eastern Cooperative Oncology Group [ECOG]7 performance status >2) are generally considered to face more risks than benefits from chemotherapy. Beyond performance status, additional factors (including organ function, comorbidity, and frailty) are also known to influence the risk of chemotherapy toxic effects.5,8 Nevertheless, decision making about chemotherapy in current practice is based on the oncologist’s “gestalt” assessment at the time of treatment initiation.
A number of studies have sought to develop more discriminative approaches for assessing the risk of chemotherapy toxic effects in individual patients.8-11 These studies have focused on different patient populations, predictors, and toxic effect outcomes, but they all showed that model-based approaches can improve risk stratification for chemotherapy toxic effects. We sought to build on these efforts to devise an efficient method for estimating the patient-specific risk of severe toxic effects from chemotherapy that could be applied to a broad population of patients receiving palliative chemotherapy for cancerous solid tumors. In this report, we describe a clinical prediction model using routinely collected clinical data to estimate the patient-specific risk of chemotherapy-related hospitalization (CRH), a surrogate for severe chemotherapy toxic effects. We derive our model in a population of patients initiating palliative chemotherapy at a community cancer center. The ability to better identify patients at elevated risk for chemotherapy toxic effects has potential to improve patient outcomes at many levels. Specifically, better risk assessment for chemotherapy toxic effects would improve the chemotherapy informed-consent process, allow for modification of treatment regimens to reduce the risk of toxic effects, and identify patients who may benefit from aggressive supportive care around the time of chemotherapy initiation.
Among 1433 patients receiving palliative chemotherapy for solid tumors, 9.2% experienced a chemotherapy-related hospitalization (CRH).
Independent predictors of CRH included age, Charlson comorbidity score, creatinine clearance, calcium level, low white blood cell count and/or platelet count, number of chemotherapy agents, and receipt of camptothecin chemotherapy.
The median predicted risk of hospitalization was 14.7% among patients who experienced a CRH, and 6.0% among patients who did not experience a CRH.
We conducted our study using a clinical registry of adult patients (age ≥18 years) initiating first-line palliative chemotherapy for malignant solid tumors between January 2003 and December 2011 (parent cohort) at the Mass General/North Shore Cancer Center (a community cancer center in northeastern Massachusetts). Malignant solid tumors were defined to exclude hematologic cancers (eg, leukemia, lymphoma, and multiple myeloma). Palliative treatment intent was documented by the treating oncologist at the time of chemotherapy initiation. The definition of chemotherapy treatment included receipt of any intravenous or oral cytotoxic or anticancer biological agent but not endocrine therapy alone.
All patients in the registry were followed up for hospitalization after initiation of chemotherapy as part of an institutional quality-assessment program.5,12 Among patients receiving chemotherapy who were subsequently hospitalized, all hospitalizations were discussed at monthly clinical meetings by a team of oncology care clinicians, including representatives from medical oncology, nursing, and pharmacy. After discussion, hospitalizations were attributed to either chemotherapy toxic effects or to causes unrelated to chemotherapy. Hospitalization was considered chemotherapy related when admission occurred within 30 days of the most recent chemotherapy administration and was judged to be definitely, probably, or possibly a result of chemotherapy treatment. Hospitalizations that were attributed to both chemotherapy toxic effects and cancer symptoms were considered to be chemotherapy-related in this analysis. Attributions were consensus based, accounting for the perspectives of all clinicians in attendance at each meeting.
We used a case-control design to structure our analysis. Case patients included all patients experiencing a CRH during first-line chemotherapy. Controls were randomly selected from 1433 patients who did not experience a chemotherapy-related hospitalization. Demographic information and data regarding putative CRH risk factors were collected retrospectively from clinical records of case and control patients. The nested case-control design was used to increase the efficiency of retrospective data collection. Data points for laboratory results and other time-varying risk factors were collected on or shortly before the date of chemotherapy initiation. The study was approved by the North Shore Medical Center institutional review board, which waived written informed consent.
We first assessed the association between CRH and each of the individual putative CRH risk factors. Demographic and clinical variables included age (continuous), sex, ECOG performance status7 (0, 1, or ≥2), Charlson comorbidity score13 (0, 1, or ≥2, excluding points for metastatic cancer), primary cancer site, body mass index (BMI; calculated as weight in kilograms divided by height in meters squared) (<18.5, 18.5-24.9, 25.0-29.9, or ≥30.0), number of chemotherapy drugs (monotherapy vs polytherapy), and chemotherapy regimen dosing (full vs reduced). Exposure to specific chemotherapy agents was defined nonexclusively as exposure to any chemotherapy agent from 1 or more of the 5 most common chemotherapy classes in our cohort, including platinums, taxanes, fluoropyrimidines, camptothecins (irinotecan or topotecan), and/or gemcitabine.
Laboratory values were assessed as either threshold or continuous variables. Threshold variables were dichotomized by the relevant upper or lower limit of the normal range, including white blood cell count (categorized as <3800/μL or >3800/μL), absolute neutrophil count (<1800/μL or >1800/μL), hemoglobin (<12 or >12 g/dL), platelet count (<150 000/μL or >150 000/μL), and levels of aspartate aminotransferase (<46 or >46 IU/L), alkaline phosphatase (<118 or >118 IU/L), and lactate dehydrogenase (<250 or >250 IU/L). Continuous laboratory variables included creatinine clearance, albumin, and calcium level. Creatinine clearance was calculated using the Cockcroft-Gault formula,14 and values for creatinine clearance exceeding 120 mL/min were adjusted down to a ceiling value of 120 mL/min (consistent with prior analysis10). Two-sided χ2 tests were used to assess relationships between CRH and categorical variables, and 2-sided t tests were used for continuous variables. Odds ratios (ORs) and 95% CIs were obtained from logistic regression models.
We then built a multivariable logistic regression model, with CRH as the dependent variable. All multivariable analyses used weights to recapitulate the overall CRH rate observed in the parent cohort; these weights represented the inverse of the probability of selection for control patients. The initial multivariable logistic regression model included all variables from the univariate analyses that were associated with CRH at a level of P < .10. In addition, age and ECOG performance status were retained in the initial multivariable model owing to their perceived clinical significance. Platelet count and white blood cell count were collapsed into a single combination variable (indicating the presence of below-normal platelet count, below-normal white blood cell count, or both) because both leucopenia and thrombocytopenia are markers of limited bone-marrow reserve. After imputation of missing data, all cases and controls were used to derive the final multivariable prediction model. Missing values for the Charlson comorbidity score (24 patients) were imputed using a multiple imputation protocol for categorical variables. Mean imputation was used for missing values of creatinine clearance (2 patients) and calcium level (2 patients).
To reduce overfitting, we built a more parsimonious prediction model from 7 of the 15 risk factors identified in the univariate screen. We used a “supervised” backward selection approach to select the predictors for the final model, driven by multivariable statistical significance (P > .05). Backward selection was influenced by clinical considerations; for example, when predictor pairs were clinically related (such as receipt of polychemotherapy and receipt of platinum chemotherapy), the order of predictor elimination was influenced by clinical reasoning in addition to multivariable statistical significance. We then tested for interaction effects between predictors retained in the final model. A significant interaction (P < .05) was identified between the Charlson comorbidity score and the combination variable for below-normal platelet count and/or below-normal white blood cell count; this interaction was added to the model.
The discrimination of the final model was evaluated by the bootstrap-adjusted C statistic, which provides a form of internal validation.15 We generated 200 bootstrap samples from the data set of 438 patient records using unrestricted random sampling with replacement. A new model was fitted to each bootstrap sample, and we then calculated the C statistic for each bootstrap model. The bootstrap-adjusted C statistic was then calculated as the crude C statistic minus the mean model optimism, where model optimism is the mean of the differences between the crude C statistic and each of the 200 bootstrap-derived C indices. Model calibration was assessed with a calibration plot. All analyses were performed using SAS software, version 9.4 (SAS Institute Inc).
The study population for this nested case-control study was selected from 1579 patients initiating first-line palliative chemotherapy for malignant solid tumors between 2003 and 2011 (Figure 1). We found CRH in 9.2% of patients from the parent cohort. Case patients in this analysis included all 146 patients who experienced CRH, and control patients included 292 randomly selected patients without CRH. The median time from chemotherapy initiation to hospitalization was 30 days among case patients, and 73% of CRHs occurred within 60 days of treatment initiation. The most common toxic effects leading to CRH were gastrointestinal (54%), infectious (27%), and hematologic (13%); multiple toxic effects were recorded for 10% of CRHs.
The demographic and clinical characteristics of case and control patients are summarized in Table 1. The median age was similar in the 2 groups (68 vs 67 years), and the most common diagnoses were cancers of the lung, gastrointestinal tract, and breast. Univariate analysis showed that CRH was significantly associated with the Charlson comorbidity score, the number of chemotherapy drugs (monotherapy vs polytherapy), and receipt of platinum, fluoropyrimidine, and/or camptothecin chemotherapy. Pretreatment clinical laboratory results are listed in Table 2. Test results were significantly associated with CRH for low white blood cell count, low platelet count, low creatinine clearance, and low levels of hemoglobin, aspartate aminotransferase, albumin, and calcium.
The final parsimonious multivariable model included 7 independent predictors of CRH (Table 3). Predictors included younger age, higher Charlson comorbidity score, lower creatinine clearance, lower calcium level, below-normal white blood cell and/or platelet count, receipt of multiple chemotherapy agents, and receipt of camptothecin chemotherapy. The receiver operating characteristic (ROC) curve is shown in eFigure 1 in the Supplement, and the crude model C statistic (the area under the ROC curve) was 0.73. The bootstrap-adjusted C statistic, which estimates the internally validated model discrimination, was 0.71 (95% CI, 0.66-0.75). The Nagelkerke pseudo r2, which describes the proportion of outcome variation explained by the model, was 0.23. The β coefficients for the final multivariable model are listed in the eTable in the Supplement.
The median predicted risk of CRH was 6.0% (interquartile range [IQR], 3.6%-11.4%) in control patients and 14.7% (IQR, 6.8%-22.5%) in case patients. The model-predicted risk distributions for case and control patients are shown in eFigure 2 in the Supplement. Model calibration is shown in the calibration plot in Figure 2. Using a CRH risk threshold of 15%, the risk prediction model identified patients experiencing CRH with a sensitivity of 49% (95% CI, 41%-57%) and a specificity of 85% (95% CI, 81%-89%). Sensitivity and specificity under alternative risk thresholds can be assessed from the ROC curve in eFigure 1 in the Supplement.
Studying a community-based cohort of patients receiving first-line palliative chemotherapy for malignant solid tumors, we created a prediction model to identify the patient-specific risk of CRH at the time of chemotherapy initiation. We found a baseline CRH risk of 9.2%; however, approximately 1 in 6 patients from the parent cohort were projected to have a CRH risk of 15% or higher. All variables in our model (including age, Charlson comorbidity score, creatinine clearance, white blood cell count, platelet count, calcium level, polychemotherapy, and exposure to camptothecin chemotherapy) are routinely collected in the course of clinical care and readily identifiable within a high-functioning electronic medical record.
A number of prior studies have presented models for predicting the risk of chemotherapy toxic effects in patients with cancer.8-11 Hurria et al8 studied elderly patients (≥65 years) with malignant solid tumors and showed that a multivariable model including geriatric assessment variables was a better predictor of toxic effects from chemotherapy than the Karnofsky performance status alone. This approach was externally validated in a population of elderly patients with lung cancer.16 Similarly, Extermann et al9 focused on elderly patients with cancer (>70 years) and developed separate models to predict hematologic and nonhematologic toxic effects from chemotherapy. Lyman and coauthors11 built a model using laboratory data and treatment characteristics to predict neutropenic complications. Finally, Hyman et al10 developed a model to identify patients with cancer participating in phase 1 clinical trials who were at high risk for serious drug-related toxic effects.
Most of these studies defined chemotherapy toxic effects in the context of the Common Terminology Criteria for Adverse Events (CTCAE).17 While the CTCAE framework defines the gold standard for adverse event reporting in clinical trials, the patient centeredness of this instrument is debatable.18 In fact, many toxic effects defined as grade 3 by the CTCAE may be entirely asymptomatic (eg, stage 2 hypertension or absolute neutrophil count <1000/μL in the absence of fever). We used a simpler and potentially more patient-centered surrogate to assess severe toxic effects from chemotherapy: CRH, an event clearly undesirable for patients and one representing an adverse event of sufficient severity and acuity to justify hospitalization. Policy makers also have a stake in reducing CRHs because acute hospitalization accounts for 48% of the cost of medical care in patients with advanced cancer.19
A number of the predictors included in our model have been previously identified as risk factors for chemotherapy toxic effects: polychemotherapy, decreased creatinine clearance, and low blood cell counts have all been included in previous toxic effects prediction models.8-11 While these risk factors may appear to be clinically self-evident, our findings reinforce that even modest deficits in renal or bone-marrow function confer measurably increased risk for toxic effects from chemotherapy. Furthermore, inclusion of multiple predictors in a single risk model can help clinicians to appreciate the combined effect of these risk factors, each of which may appear to be of negligible salience individually.
Anticipated associations of CRH with poor ECOG performance status and full-dose (vs reduced-dose) chemotherapy were not found to be statistically significant. This should not be interpreted to suggest that performance status and chemotherapy dose are not associated with the risk for chemotherapy toxic effects. Rather, our findings suggest that oncologists are already incorporating performance status and chemotherapy dose intensity into decisions about chemotherapy treatment in ways that limit toxic effects. In addition, our model showed an inverse association between age and CRH risk. This finding seems to belie the presumption that age is positively associated with the risk for chemotherapy toxic effects. However, this association must be interpreted in the context of the multivariable analytic approach, under the assumption that other risk factors are held constant with advancing age. In actuality, many of the risk factors in the model (particularly comorbidity, renal function, and bone-marrow function) are themselves associated with age. As such, individual coefficients of the multivariable model should be interpreted with caution.
Our study has limitations. We used a qualitative consensus review process to distinguish CRHs from hospitalizations attributable to other causes, which complicates replication of our approach. Nevertheless, we found that CRHs generally occurred within 60 days of chemotherapy initiation, supporting a cause-and-effect relationship between chemotherapy administration and clinician-adjudicated CRH. Our findings are subject to time-window bias, whereby patients with longer duration of chemotherapy would be more likely to eventually experience CRH.20 Exposure to this bias is reduced by the observation that nearly three-quarters of the CRHs in our study occurred within 60 days of chemotherapy initiation. Our study included patients with solid tumors initiating first-line palliative chemotherapies but may not generalize well to patients initiating later-line palliative chemotherapy or to patients with malignant hematologic conditions. We did not include geriatric assessment variables in our study, although other investigators have demonstrated that measures of frailty from geriatric assessment are associated with increased risk of chemotherapy toxic effects.8,9,16,21 However, geriatric assessments are not commonly performed in many clinical practices, most likely owing to perceived resource and time constraints. Finally, the model-building process was subjective, incorporating both clinical reasoning and multivariable statistical significance. Alternative model-building approaches would likely lead to different model specifications.
Strategies to harness the latent power of routinely collected clinical data have been proposed as an efficient and cost-effective way to improve health care quality.22,23 Because our model used data that are available from within structured fields in electronic health records, our approach could be used to create an automated clinical decision support tool at the point of chemotherapy order entry, without requiring additional data entry. In this context, an externally validated prediction model could be used by physicians to counsel patients about their individual-level risk of CRH during shared decision making around initiation of chemotherapy. In addition, physicians could use CRH risk estimates to justify modification of chemotherapy regimens to reduce the risk of toxic effects. Patients at substantially elevated risk of CRH could be enrolled in case management programs to more effectively deliver aggressive supportive care to those most likely to benefit from such services.
In conclusion, we sought to develop a prediction model to estimate the risk of CRH in patients receiving palliative chemotherapy. Using routinely collected clinical data, our model identified a subpopulation of patients at substantially elevated risk for CRH. Model discrimination, characterized by an internally validated C statistic of 0.71, was comparable to the discrimination of previously developed models that required collection of nonstandard data elements.8,9 Our approach shows promise for developing a tool that can help patients and clinicians reduce the risk of chemotherapy toxic effects (and hospitalizations) in an individualized and patient-centered fashion. We are currently pursuing opportunities for further development of the CRH prediction model.
Accepted for Publication: March 13, 2015.
Corresponding Author: Gabriel A. Brooks, MD, Dana-Farber Cancer Institute, 450 Brookline Ave, Boston, MA 02215 (firstname.lastname@example.org).
Published Online: April 30, 2015. doi:10.1001/jamaoncol.2015.0828.
Author Contributions: Drs Brooks and Jacobson had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Drs Brooks and Kansagra contributed equally to this article.
Study concept and design: Kansagra, Linden, Jacobson.
Acquisition, analysis, or interpretation of data: Brooks, Kansagra, Rao, Weitzman, Linden, Jacobson.
Drafting of the manuscript: Brooks, Kansagra, Rao, Linden,
Critical revision of the manuscript for important intellectual content: Brooks, Kansagra, Rao, Weitzman, Linden, Jacobson.
Statistical analysis: Brooks, Kansagra, Rao, Linden,
Obtained funding: Linden, Jacobson.
Administrative, technical, or material support: Kansagra, Weitzman, Linden, Jacobson.
Study supervision: Weitzman, Linden, Jacobson.
Conflict of Interest Disclosures: None reported.
Funding/Support: This research was supported by a grant from the Read Family Trust. Dr Brooks was supported by a Young Investigator Award from the Conquer Cancer Foundation of the American Society of Clinical Oncology and by program grant R25CA09220 from the National Cancer Institute of the National Institutes of Health.
Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; or preparation, review, or approval of the manuscript.
Disclaimer: Contents of this publication are the sole responsibility of the authors, and do not necessarily represent the official views of the National Cancer Institute, the National Institutes of Health, the Department of Veterans Affairs, or the American Society of Clinical Oncology.
Previous Presentation: This research was presented at the American Society of Clinical Oncology Quality Care Symposium; November 1 and 2, 2013; San Diego, California.
Create a personal account or sign in to: