[Skip to Content]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Figure 1.  Flow Diagram of the Development and Validation Cohorts
Flow Diagram of the Development and Validation Cohorts
Figure 2.  Calibration Plots for Years 0 and 4
Calibration Plots for Years 0 and 4

Dots represent the deciles of patients’ observed 1-year probability of death plotted against their predicted 1-year probability of death.

Table 1.  Baseline Characteristics of the Study Cohort at Years 0 and 4a
Baseline Characteristics of the Study Cohort at Years 0 and 4a
Table 2.  Final Time-to-Death Model for Developmental Cohort Following Backward Eliminationa
Final Time-to-Death Model for Developmental Cohort Following Backward Eliminationa
1.
Bakitas  M, Lyons  KD, Hegel  MT,  et al.  Effects of a palliative care intervention on clinical outcomes in patients with advanced cancer: the Project ENABLE II randomized controlled trial.  JAMA. 2009;302(7):741-749. doi:10.1001/jama.2009.1198PubMedGoogle ScholarCrossref
2.
Zimmermann  C, Swami  N, Krzyzanowska  M,  et al.  Early palliative care for patients with advanced cancer: a cluster-randomised controlled trial.  Lancet. 2014;383(9930):1721-1730. doi:10.1016/S0140-6736(13)62416-2PubMedGoogle ScholarCrossref
3.
Temel  JS, Greer  JA, Muzikansky  A,  et al.  Early palliative care for patients with metastatic non-small-cell lung cancer.  N Engl J Med. 2010;363(8):733-742. doi:10.1056/NEJMoa1000678PubMedGoogle ScholarCrossref
4.
Ferrell  BR, Temel  JS, Temin  S, Smith  TJ.  Integration of palliative care into standard oncology care: ASCO clinical practice guideline update summary.  J Oncol Pract. 2017;13(2):119-121. doi:10.1200/JOP.2016.017897PubMedGoogle ScholarCrossref
5.
National Hospice and Palliative Care Organization.  NHPCO Facts and Figures: Hospice Care in America. National Hospice and Palliative Care Organization; 2016.
6.
Rabin  BA, Gaglio  B, Sanders  T,  et al.  Predicting cancer prognosis using interactive online tools: a systematic review and implications for cancer care providers.  Cancer Epidemiol Biomarkers Prev. 2013;22(10):1645-1656. doi:10.1158/1055-9965.EPI-13-0513PubMedGoogle ScholarCrossref
7.
Zhi  WI, Smith  TJ.  Early integration of palliative care into oncology: evidence, challenges and barriers.  Ann Palliat Med. 2015;4(3):122-131.PubMedGoogle Scholar
8.
Fassbender  K, Watanabe  SM.  Early palliative care and its translation into oncology practice in Canada: barriers and challenges.  Ann Palliat Med. 2015;4(3):135-149.PubMedGoogle Scholar
9.
Cheon  S, Agarwal  A, Popovic  M,  et al.  The accuracy of clinicians’ predictions of survival in advanced cancer: a review.  Ann Palliat Med. 2016;5(1):22-29.PubMedGoogle Scholar
10.
Chow  E, Harth  T, Hruby  G, Finkelstein  J, Wu  J, Danjoux  C.  How accurate are physicians’ clinical predictions of survival and the available prognostic tools in estimating survival times in terminally ill cancer patients? a systematic review.  Clin Oncol (R Coll Radiol). 2001;13(3):209-218. doi:10.1007/s001740170078PubMedGoogle Scholar
11.
Sutradhar  R, Rostami  M, Barbera  L.  Patient-reported symptoms improve performance of risk prediction models for emergency department visits among patients with cancer: a population-wide study in Ontario using administrative data.  J Pain Symptom Manage. 2019;58(5):745-755. doi:10.1016/j.jpainsymman.2019.07.007PubMedGoogle ScholarCrossref
12.
Seow  H, Barbera  L, Sutradhar  R,  et al.  Trajectory of performance status and symptom scores for patients with cancer during the last six months of life.  J Clin Oncol. 2011;29(9):1151-1158. doi:10.1200/JCO.2010.30.7173PubMedGoogle ScholarCrossref
13.
Maltoni  M, Caraceni  A, Brunelli  C,  et al; Steering Committee of the European Association for Palliative Care.  Prognostic factors in advanced cancer patients: evidence-based clinical recommendations: a study by the Steering Committee of the European Association for Palliative Care.  J Clin Oncol. 2005;23(25):6240-6248. doi:10.1200/JCO.2005.06.866PubMedGoogle ScholarCrossref
14.
Stillwell  AP, Ho  YH, Veitch  C.  Systematic review of prognostic factors related to overall survival in patients with stage IV colorectal cancer and unresectable metastases.  World J Surg. 2011;35(3):684-692. doi:10.1007/s00268-010-0891-8PubMedGoogle ScholarCrossref
15.
Barbera  L, Seow  H, Howell  D,  et al.  Symptom burden and performance status in a population-based cohort of ambulatory cancer patients.  Cancer. 2010;116(24):5767-5776. doi:10.1002/cncr.25681PubMedGoogle ScholarCrossref
16.
Guthrie  DM, Harman  LE, Barbera  L,  et al.  Quality indicator rates for seriously ill home care clients: analysis of resident assessment instrument for home care data in six Canadian provinces.  J Palliat Med. 2019;22(11):1346-1356. doi:10.1089/jpm.2019.0022PubMedGoogle ScholarCrossref
17.
Collins  GS, Reitsma  JB, Altman  DG, Moons  KG.  Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement.  BMJ. 2015;350:g7594. doi:10.1136/bmj.g7594PubMedGoogle ScholarCrossref
18.
Anderson  F, Downing  GM, Hill  J, Casorso  L, Lerch  N.  Palliative performance scale (PPS): a new tool.  J Palliat Care. 1996;12(1):5-11. doi:10.1177/082585979601200102PubMedGoogle ScholarCrossref
19.
Chang  VT, Hwang  SS, Feuerman  M.  Validation of the Edmonton Symptom Assessment Scale.  Cancer. 2000;88(9):2164-2171. doi:10.1002/(SICI)1097-0142(20000501)88:9<2164::AID-CNCR24>3.0.CO;2-5PubMedGoogle ScholarCrossref
20.
Canadian Quality Council of Ontario. Cancer System Quality Index. 2016. Accessed October 25, 2012. https://www.csqi.on.ca/
21.
Popovic  G, Harhara  T, Pope  A,  et al.  Patient-reported functional status in outpatients with advanced cancer: correlation with physician-reported scores and survival.  J Pain Symptom Manage. 2018;55(6):1500-1508. doi:10.1016/j.jpainsymman.2018.02.015PubMedGoogle ScholarCrossref
22.
Martin  L, Watanabe  S, Fainsinger  R,  et al.  Prognostic factors in patients with advanced cancer: use of the patient-generated subjective global assessment in survival prediction.  J Clin Oncol. 2010;28(28):4376-4383. doi:10.1200/JCO.2009.27.1916PubMedGoogle ScholarCrossref
23.
Hirdes  JP, Ljunggren  G, Morris  JN,  et al.  Reliability of the interRAI suite of assessment instruments: a 12-country study of an integrated health information system.  BMC Health Serv Res. 2008;8:277. doi:10.1186/1472-6963-8-277PubMedGoogle ScholarCrossref
24.
Morris  JN, Fries  BE, Steel  K,  et al.  Comprehensive clinical assessment in community setting: applicability of the MDS-HC.  J Am Geriatr Soc. 1997;45(8):1017-1024. doi:10.1111/j.1532-5415.1997.tb02975.xPubMedGoogle ScholarCrossref
25.
Poss  JW, Jutan  NM, Hirdes  JP,  et al.  A review of evidence on the reliability and validity of Minimum Data Set data.  Healthc Manage Forum. 2008;21(1):33-39. doi:10.1016/S0840-4704(10)60127-5PubMedGoogle ScholarCrossref
26.
Barbera  L, Sussman  J, Viola  R,  et al.  Factors associated with end-of-life health service use in patients dying of cancer.  Healthc Policy. 2010;5(3):e125-e143. doi:10.12927/hcpol.2013.21644PubMedGoogle Scholar
27.
Hirdes  JP, Frijters  DH, Teare  GF.  The MDS-CHESS scale: a new measure to predict mortality in institutionalized older people.  J Am Geriatr Soc. 2003;51(1):96-100. doi:10.1034/j.1601-5215.2002.51017.xPubMedGoogle ScholarCrossref
28.
Muggah  E, Graves  E, Bennett  C, Manuel  DG.  The impact of multiple chronic diseases on ambulatory care use; a population based study in Ontario, Canada.  BMC Health Serv Res. 2012;12:452. doi:10.1186/1472-6963-12-452PubMedGoogle ScholarCrossref
29.
Muggah  E, Graves  E, Bennett  C, Manuel  DG.  Ascertainment of chronic diseases using population health data: a comparison of health administrative data and patient self-report.  BMC Public Health. 2013;13:16. doi:10.1186/1471-2458-13-16PubMedGoogle ScholarCrossref
30.
Agopian  VG, Harlander-Locke  M, Zarrinpar  A,  et al.  A novel prognostic nomogram accurately predicts hepatocellular carcinoma recurrence after liver transplantation: analysis of 865 consecutive liver transplant recipients.  J Am Coll Surg. 2015;220(4):416-427. doi:10.1016/j.jamcollsurg.2014.12.025PubMedGoogle ScholarCrossref
31.
Kawai  K, Ishihara  S, Yamaguchi  H,  et al.  Nomograms for predicting the prognosis of stage IV colorectal cancer after curative resection: a multicenter retrospective study.  Eur J Surg Oncol. 2015;41(4):457-465. doi:10.1016/j.ejso.2015.01.026PubMedGoogle ScholarCrossref
32.
Harrell  F.  Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis. Springer-Verlag; 2001. doi:10.1007/978-1-4757-3462-1
33.
O’Brien  BA, Cohen  RJ, Ryan  A, Sengupta  S, Mills  J.  A new preoperative nomogram to predict minimal prostate cancer: accuracy and error rates compared to other tools to select patients for active surveillance.  J Urol. 2011;186(5):1811-1817. doi:10.1016/j.juro.2011.06.060PubMedGoogle ScholarCrossref
34.
Therneau  TM, Grambsch  PM.  Modeling Survival Data: Extending the Cox Model. Springer; 2000. doi:10.1007/978-1-4757-3294-8
35.
Yi  M, Meric-Bernstam  F, Kuerer  HM,  et al.  Evaluation of a breast cancer nomogram for predicting risk of ipsilateral breast tumor recurrences in patients with ductal carcinoma in situ after local excision.  J Clin Oncol. 2012;30(6):600-607. doi:10.1200/JCO.2011.36.4976PubMedGoogle ScholarCrossref
36.
Balachandran  VP, Gonen  M, Smith  JJ, DeMatteo  RP.  Nomograms in oncology: more than meets the eye.  Lancet Oncol. 2015;16(4):e173-e180. doi:10.1016/S1470-2045(14)71116-7PubMedGoogle ScholarCrossref
37.
Su  J, Barbera  L, Sutradhar  R.  Do repeated assessments of performance status improve predictions for risk of death among patients with cancer? a population-based cohort study.  Palliat Med. 2015;29(6):547-553. doi:10.1177/0269216314568231PubMedGoogle ScholarCrossref
38.
PROVIEW calculator. Accessed March 3, 2020. www.individualizedhealth.ca
39.
Wishart  GC, Bajdik  CD, Dicks  E,  et al.  PREDICT Plus: development and validation of a prognostic model for early breast cancer that includes HER2.  Br J Cancer. 2012;107(5):800-807. doi:10.1038/bjc.2012.338PubMedGoogle ScholarCrossref
40.
Kattan  MW, Cuzick  J, Fisher  G,  et al; Transatlantic Prostate Group.  Nomogram incorporating PSA level to predict cancer-specific survival for men with clinically localized prostate cancer managed without curative intent.  Cancer. 2008;112(1):69-74. doi:10.1002/cncr.23106PubMedGoogle ScholarCrossref
41.
Stephenson  AJ, Kattan  MW, Eastham  JA,  et al.  Prostate cancer-specific mortality after radical prostatectomy for patients treated in the prostate-specific antigen era.  J Clin Oncol. 2009;27(26):4300-4305. doi:10.1200/JCO.2008.18.2501PubMedGoogle ScholarCrossref
42.
Isenberg  SR, Crossnohere  NL, Patel  MI,  et al.  An advance care plan decision support video before major surgery: a patient- and family-centred approach.  BMJ Support Palliat Care. 2018;8(2):229-236. doi:10.1136/bmjspcare-2017-001449PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Original Investigation
    Oncology
    April 1, 2020

    Development and Validation of a Prognostic Survival Model With Patient-Reported Outcomes for Patients With Cancer

    Author Affiliations
    • 1Department of Oncology, McMaster University, Hamilton, Ontario, Canada
    • 2Institute for Clinical Evaluative Sciences, Toronto, Ontario, Canada
    • 3Division of Palliative Care, Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada
    • 4Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
    • 5Department of Oncology, University of Calgary, Calgary, Alberta, Canada
    • 6Tom Baker Cancer Centre, Alberta Health Services, Calgary, Alberta, Canada
    • 7Canadian Partnership Against Cancer, Toronto, Ontario, Canada
    • 8Department of Kinesiology and Physical Education, Department of Health Sciences, Wilfrid Laurier University, Waterloo, Ontario, Canada
    • 9Temmy Latner Centre for Palliative Care, Lunenfeld Tanenbaum Research Institute, Sinai Health System, Toronto, Ontario, Canada
    • 10Division of Palliative Care, Department of Family and Community Medicine, University of Toronto, Toronto, Ontario, Canada
    • 11University of Ottawa School of Epidemiology and Public Health, Ottawa, Ontario, Canada
    • 12Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada
    JAMA Netw Open. 2020;3(4):e201768. doi:10.1001/jamanetworkopen.2020.1768
    Key Points español 中文 (chinese)

    Question  Is it possible to develop a risk prediction model of survival for patients with cancer that incorporates patient-reported outcomes over time?

    Findings  In this prognostic study of data from 255 494 patients with cancer, the mean) time to death from diagnosis was 567 days. The model found that the following factors were associated with increased risk of death by more than 10%: being hospitalized; having congestive heart failure, chronic obstructive pulmonary disease, or dementia; having moderate to high pain; having worse well-being; having functional status in the transitional or end-of-life phase; having any problems with appetite; receiving end-of-life home care; and living in a nursing home.

    Meaning  Patients and families may be more informed when making decisions, such as about treatments or palliative care, if they can easily calculate prognostic information.

    Abstract

    Importance  Existing prognostic cancer tools include biological and laboratory variables. However, patients often do not know this information, preventing them from using the tools and understanding their prognosis.

    Objective  To develop and validate a prognostic survival model for all cancer types that incorporates information on symptoms and performance status over time.

    Design, Setting, and Participants  This is a retrospective, population-based, prognostic study of data from patients diagnosed with cancer from January 1, 2008, to December 31, 2015, in Ontario, Canada. Patients were randomly selected for model derivation (60%) and validation (40%). The derivation cohort was used to develop a multivariable Cox proportional hazards regression model with baseline characteristics under a backward stepwise variable selection process to predict the risk of mortality as a function of time. Covariates included demographic characteristics, clinical information, symptoms and performance status, and health care use. Model performance was assessed on the validation cohort by C statistics and calibration plots. Data analysis was performed from February 6, 2018, to November 6, 2019.

    Main Outcomes and Measures  Time to death from diagnosis (year 0) recalculated at each of 4 annual survivor marks after diagnosis (up to year 4).

    Results  A total of 255 494 patients diagnosed with cancer were identified (135 699 [53.1%] female; median age, 65 years [interquartile range, 55-73 years]). The cohort decreased to 217 055, 184 822, 143 649, and 109 569 patients for each of the 4 years after diagnosis. In the derivation cohort year 0, and the most common cancers were breast (30 855 [20.1%]), lung (19 111 [12.5%]), and prostate (18 404 [12.0%]). A total of 47 614 (31.1%) had stage III or IV disease. The mean (SD) time to death in year 0 was 567 (715) days. After backward stepwise selection in year 0, the following factors were associated with increased risk of death by more than 10%: being hospitalized; having congestive heart failure, chronic obstructive pulmonary disease, or dementia; having moderate to high pain; having worse well-being; having functional status in the transitional or end-of-life phase; having any problems with appetite; receiving end-of-life home care; and living in a nursing home. Model discrimination was high for all models (C statistic: 0.902 [year 0], 0.912 [year 1], 0.912 [year 2], 0.909 [year 3], and 0.908 [year 4]).

    Conclusions and Relevance  The model accurately predicted changing cancer survival risk over time using clinical, symptom, and performance status data and appears to have the potential to be a useful prognostic tool that can be completed by patients. This knowledge may support earlier integration of palliative care.

    Introduction

    Several randomized clinical trials demonstrated the benefits of early integration of palliative care (ie, at diagnosis) with active cancer treatment, such as improved quality of life and symptom control.1-3 This evidence led the American Society of Clinical Oncology to endorse the provision of early palliative care concurrently with standard oncologic care.4 Although evidence indicates the positive effect of palliative care integration at the time of diagnosis, patients generally receive palliative care close to death or not at all. Data from the US5 indicate that palliative care was accessed in 45% of all deaths for a median of 17 days before death. An increasing body of research has focused on developing prognostic tools, particularly online tools, to help practitioners predict death in patients with cancer. A systematic review identified 22 online prognostic tools that addressed 89 different cancers.6 However, prognostic tools largely fail to integrate palliative care earlier in the disease trajectory for several reasons.

    First, most tools were designed for use by oncologists. However, even with prognostic tools, oncologists often struggle with when to discuss prognosis and palliative care because they do not want to take away patient hope, and cancer advancements have increased treatment options and clinical trials.7,8 When practitioners discuss prognosis, research shows their predictions are overly optimistic.9,10 Second, existing tools have limitations in their usefulness as the disease progresses. Many of the tools predict death from diagnosis but do not account for changes over time, such as in treatment plan or health services use. Most tools also do not incorporate patient-reported outcomes, such as performance status or symptom burden, which have clinical and statistical prognostic value across multiple cancers.11,12 Third, patients face barriers to use the tools directly. Systematic reviews13,14 have found that many prognostic tools require biological and laboratory variables, such as cancer antigen levels, elevated C-reactive protein level, and leukocytosis, which are not typically known by patients. This requirement prevents patients from obtaining prognostic information that could help them initiate discussions about palliative care.

    In this study, we aimed to develop and validate a prognostic model to predict survival in patients with cancer. To address prior limitations, the model uses easily known clinical information and incorporates patient-reported outcomes (ie, common symptoms and performance status) over time using unique databases available in Ontario, Canada.15,16 Thus, the model can be completed by patients and families. We named the model PROVIEW by blending the goal to help patients preview the future to be proactive. PROVIEW aims to provide changing survival predictions as the disease progresses over time, which could support discussions about integrating palliative care even alongside disease-modifying therapies.

    Methods
    Study Design and Population

    We performed a population-based, retrospective prognostic study of data from adults diagnosed with cancer, as confirmed by the provincial cancer registry in Ontario, Canada, from January 1, 2008, to December 31, 2015. Data analysis was performed from February 6, 2018, to November 6, 2019. The study was reviewed by Hamilton Integrated Research Ethics Board and deemed exempt because it was a deidentified, secondary data analysis. This study followed the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) reporting guideline.17

    Data Sources

    We used the following linked administrative databases (and corresponding covariates): (1) Ontario Cancer Registry (cancer type, diagnosis date, and stage); (2) Vital Statistics (age, sex, and date of death); (3) Statistics Canada (rurality, income quintile, and region); (4) Activity Level Reporting (chemotherapy and radiation treatment); (5) Discharge Abstract Database (hospitalization dates, diagnoses, surgery for cancer, and comorbidity); (6) National Acute Care Registry System (emergency department visits and reasons); (7) physician billing (physician visits and billing codes); (8) Home Care database (nursing and personal support); (9) Symptom Management database (symptoms and performance status); and (10) interRAI database (performance status and symptoms).

    The 2 databases that contain population-based performance status and symptom data are the Symptom Management and interRAI databases. The Symptom Management database began in 2007 when Cancer Care Ontario mandated the systematic screening of outpatients with cancer for symptoms using the Edmonton Symptom Assessment System (ESAS) and for performance status using the Palliative Performance Scale (PPS).15 Every patient being treated at a cancer center is eligible to complete the ESAS and PPS, both of which are validated tools in populations with cancer.18,19 The monthly provincial screening completion rate is 56%.20 The ESAS asks patients to self-report the severity of 9 symptoms (ie, pain, depression, well-being, shortness of breath, anxiety, nausea, tiredness, drowsiness, and appetite) on a scale of 0 (symptom absent) to 10 (most severe), whereas the PPS describes a patient’s performance status based on a patient’s level of ambulation, level of activity, and ability to perform self-care. The PPS is scored from 0 to 100 (in 10-point increments), with 80 to 100 indicating stable, 40 to 70 indicating transitional, 10 to 30 indicating end of life, and 0 indicating dead. The PPS is completed by the practitioner during the patient’s visit. In 2013, Ontario also began collecting functional scores using a patient-completed Eastern Cooperative Oncology Group score, which is comparable and highly correlated with the physician-reported PPS.21,22

    The interRAI database began in 2002, when Ontario mandated the use of the Resident Assessment Instrument for Home Care, a standardized tool for patients receiving publicly funded home care services for an expected 60 days or more. The assessment is akin to the Minimum Data Set used internationally and is valid and reliable.23-25 Seventy percent of patients with cancer use home care in the last year of life.26 The assessment collects quality-of-life data for approximately 300 unique items that measure domains, such as the presence of moderate to severe pain or depression, presence of caregiver living in patient’s home, and performance status via the health instability CHESS scale (change in decision-making, change in activities of daily living status, and end-stage disease).27 The assessment is completed by the case manager at intake and reassessed at least every 6 months.

    Outcome

    The primary outcome was time to death (days) per date of death in the Vital Statistics database. The initial index date for each patient was the date of diagnosis. Because covariates and treatments may change over time, we also aimed to predict conditional survival probabilities; thus, prediction models were redeveloped by moving the index date to the 1-, 2-, 3-, and 4-year survival marks. Only patients who were alive at those marks contributed to each corresponding conditional analysis. All covariates were recalculated at each new index date to avoid incorporating time-varying covariates into the regression model because predictions are meant to be based on information known only at the current time.

    Covariates

    Each model included the following baseline covariates: demographic characteristics (age at diagnosis, sex, caregiver living with the patient [yes or no], and lives within 50 km of a cancer center [yes or no]); clinical data (diagnosis date, cancer type, cancer stage, presence of 1 of 13 other chronic diseases as determined by validated algorithms,28,29 type of chemotherapy [publicly funded oral drugs, immunotherapy, and systemic agents], receipt of radiation treatment [yes or no], and/or cancer surgery [yes or no] in the past [from diagnosis up to 3 months previously] and recently [within the past 3 months]); patient-reported outcomes (performance status and 9 symptom scores within 3 months of index date); and health care use within 3 months of the index date (prior hospitalization, hospitalizations for palliative care [including palliative care consultation], living in long-term care, receipt of end-of-life home care services, having a regular family physician, and received physician home visit).

    Statistical Analysis

    For the primary outcome of time to death, prediction methods for time-to-event data were implemented starting from diagnosis and then reimplemented at each of the 4 yearly survival marks after diagnosis. Each derived prediction model followed the below steps.

    Developing the Prediction Model

    We randomly selected 60% of eligible patients for model derivation and used the other 40% for validation. To ensure random sampling, we assessed and compared the distribution of baseline characteristics between the derivation and validation cohorts. Using the derivation cohort, we used a multivariable Cox proportional hazards regression model with baseline (time-fixed) characteristics to predict the hazard of mortality as a function of time. A priori, we created a multivariable model that consisted of all potential variables mentioned above. We then used the backward stepwise selection procedure for variable selection with a liberal 2-sided P < .15 as the retention criteria.30 The proportional hazards assumption was assessed by including interactions with time and each covariate into the model. We centered continuous covariates, such as age, and explored linear and quadratic terms. Missing data from patient-reported categorical variables were handled by creating an additional missing category for that variable. Most of the missing data were attributable to patients not completing an ESAS, although occasionally patients skip certain questions on the ESAS. Because there was no obvious missing pattern, we elected to create a missing category rather than to impute or remove these patients from the analysis. Interactions between cancer type and stage were also incorporated with a goal of achieving maximal discriminative ability within the derivation cohort,31,32 as determined by the concordance index.33

    Validating the Prediction Model

    After the final regression model was established, the 1-year predicted probability of death was calculated for each patient in the validation cohort based on their specific covariate values, the estimates of the regression variables from step 1, and the estimate of the baseline survival function from step 1.34 Calibration (how close the model-estimated risk is to the observed risk) was examined by grouping patients into deciles of model-estimated 1-year risk of death. We then reviewed the plot of the observed against the predicted 1-year probabilities of death for patients in each decile.35

    We measured the model’s discriminative ability (ability to distinguish between patients who died from those who did not die) via a concordance index (C index).36,37 Concordance for survival data was calculated as the proportion of pairs in which the patient who died had a higher predicted probability than the patient who did not die. All analyses were conducted using the statistical software R, version 2.15 (R Project for Statistical Computing) and SAS, version 9.3 (SAS Institute Inc).

    Results

    We identified 255 494 patients (135 699 [53.1%] female; median age, 65 years [interquartile range, 55-73 years]) diagnosed with cancer during 2008 to 2015. Because we repeated the derivation and validation process each year up to 4 years after diagnosis conditional on survival, the total cohort decreased to 217 055 in year 1, 184 822 in year 2, 143 649 in year 3, and 109 569 in year 4 (Figure 1). We randomly split each total cohort into derivation (60%) and validation cohorts (40%). Characteristics between the derivation and validation cohort were nearly identical in the diagnosis year (year 0).(Table 1). In the derivation cohort year 0, the most common cancers were breast (30 855 [20.1%]), lung (19 111 [12.5%]), prostate (18 404 [12.0%]), and colorectal (16 776 [10.9%]). A total of 47 614 (31.1%) of the cohort had stage III or IV disease, 66 958 (43.7%) had stage I or II disease, and 38 724 (25.2%) had unknown stage in the registry. Within the first 3 months of diagnosis, 71 479 (46.6%) had cancer-related surgery, 41 486 (27.1%) received chemotherapy, and 37 581 (24.5%) received radiation therapy. Although half of the patients did not have a performance status recorded within 3 months of diagnosis, 13 320 (8.7%) were in the transitional stage and 1 709 (1.1%) were at end of life. A total of 23 818 (15.5%) of the cohort had moderate to high pain, 43 879 (28.6%) had no pain, and 62 049 (40.5%) had missing values. Within 3 months of diagnosis, 10 172 (6.6%) were hospitalized for palliative care intent, and 9 038 (5.9%) received end-of-life home care services. The main difference between the year 4 and year 0 derivation and validation cohorts was that fewer patients were still receiving treatment in year 4 (eTable 1, eTable 2, eFigure 1, and eFigure 2 in the Supplement include all variables across all years and additional analyses).

    After backward stepwise selection, each yearly survival model had a different set of variables included in the final prediction model (Table 2). In the year 0 model, the following factors were associated with increased instantaneous risk of death by more than 10%: having lung cancer; having worse than stage I disease; being hospitalized for any reason; and, especially if the main reason was for palliative care, having congestive heart failure, chronic obstructive pulmonary disease, or dementia; having moderate or high pain; having worse well-being; having a performance status in the transitional or end-of-life phase; having any problems with appetite; receiving end-of-life home care; and living in a nursing home.

    Figure 2 gives calibration plots for year 0 and year 4 in the validation cohorts. Model discrimination in the validation cohorts was high. The C index for the 5 yearly models was 0.902 (year 0), 0.912 (year 1), 0.912 (year 2), 0.909 (year 3), and 0.908 (year 4).

    To exemplify how the model could be used, we consider the following hypothetical scenario. A 70-year-old man was diagnosed with stage III lung cancer 2 years ago (ie, the calculator would use the year 2 model). His baseline characteristics at year 2 were that he received chemotherapy and radiation therapy in the past (ie, between diagnosis until 3 months ago) and received chemotherapy recently (ie, within the past 3 months) but stopped receiving radiation therapy recently. He had no other chronic conditions, no symptoms except a score of 10 (severe) for worst appetite, and a performance status score of 60 (transitional). For someone with these baseline characteristics in our model, the probability of surviving another 365 days would be 82.4% (95% CI, 80.2%-84.6%) and another 1825 days (5 more years) would be 23.4% (95% CI, 19.2%-28.6%). If the man was hospitalized shortly thereafter, with the use of the same baseline characteristics in the model except indicating a “yes” for a recent hospitalization, the probability of surviving another 365 days would be 71.1% (95% CI, 67.7%-74.5%). If the man experienced adverse effects of chemotherapy and wondered how stopping chemotherapy would affect his long-term survival, with the use of the same baseline characteristics except indicating a “no” for recent chemotherapy, the probability of surviving another 365 days would be 90.1% (95% CI, 88.8%-91.4%). If everything stayed the same and the man lived to 3 years after diagnosis, the probability of surviving another 365 days to year 4 would be 83.8% (95% CI, 81.0%-86.7%) and another 1825 days (to year 8) would be 24.0% (95% CI, 18.3%-31.6%). A first iteration of the PROVIEW calculator is available online.38

    Discussion

    In this study, we developed and validated a predictive survival model that can be used for all cancer types and incorporates patient-reported outcomes of performance status and symptom severity. By using a large population-based cohort, we achieved high calibration and discrimination. To our knowledge, PROVIEW is the only cancer prognostic model that uses these patient-reported outcomes and updates the risk yearly after diagnosis. Because the covariates are self-reportable by patients and predict risk in days, the model has potential to be a patient-completed online tool, allowing patients to examine survival predictions during various periods as their condition changes.

    Compared with other online prognostic tools, such as the UK’s PREDICT for breast cancer39 or prostate cancer nomograms,40 PROVIEW has features that may make the model easier for patients to use. PROVIEW uses only variables that are easily reported by the patient, whereas other tools require clinical knowledge (eg, biomarkers), which may not be known by patients. For instance, PREDICT uses ERBB2 (formerly HER2 or HER2/neu) status, estrogen receptor status, and tumor size; the prostate cancer nomograms require knowledge of prostate-specific antigen levels, biopsy cores, or pathology reports. In addition, some tools report mean life expectancy or survival at predetermined periods (eg, 5- and 10-year survival from diagnosis), whereas PROVIEW models survival in days, allowing the user to choose long or short periods in the future. For example, patients at the end of life may be more interested in 30-day survival than 5-year survival. Moreover, because the model was recalculated at each 1-year anniversary and includes symptoms and performance status, it can be used at any time within the first 5 years after diagnosis and accounts for changes in a patient’s condition over time. Some tools have different versions for various posttreatment phases (eg, after radical prostatectomy),41 but they do not differentiate among individuals who had the same treatments but have drastically different performance status.

    The hypothetical case example described earlier gives potential scenarios in which the model may be useful to inform decision-making and initiate palliative care discussions earlier. In the scenario in which the patient was hospitalized, the 1-year survival risk would decrease. This change in predicted survival may trigger patients and families to review the general outlook of disease trajectory with practitioners, which may lead to discussions about palliative care even though death is not imminent. In the scenario in which the patient considers stopping chemotherapy, the 1-year survival risk would increase. This increase may be associated with the confounding fact that patients who stop chemotherapy might have responded well to treatment, achieved remission, and thus live longer. These nuances need to be discussed with practitioners, along with clinical factors that are not available in the model and preferences and goals of care. Patients can use the model’s survival predictions, which uniquely incorporate changes in symptoms, performance status, treatment, and hospital use along the disease trajectory, to inform discussions and improve decision-making with practitioners.

    Limitations

    This study has limitations. Data were not available on genetic biomarkers and specific targeted therapies, which would increase the accuracy of our predictions, particularly for cancer-specific models. Symptom and performance status data at various time points were missing because some patients chose not to voluntarily report them at cancer centers or they did not receive home care assessments and services. Nonetheless, the largest, longitudinal, population-based databases with this information were used. In this version, worsening symptoms and performance status were not considered as outcomes or how they could be modified by other variables. This analysis is planned as a subsequent step, which would further support the model’s usefulness for early palliative care integration. Although the model was validated and the initial online calculator is available, an important next step is to test, validate, and refine the online tool with patient and family users.42

    Conclusions

    The PROVIEW model appeared to accurately predict changing cancer survival risk over time using administrative clinical data and patient-reported outcomes of symptoms and performance status. Because the model covariates can be completed by patients, PROVIEW may be a useful patient-facing online tool, allowing them to prepare questions around goals of care and treatment preferences before an oncologist visit. In this way, PROVIEW could help patients and families initiate conversations with practitioners about the changing disease trajectory and explore the benefits of palliative care supports earlier.

    Back to top
    Article Information

    Accepted for Publication: January 30, 2020.

    Published: April 1, 2020. doi:10.1001/jamanetworkopen.2020.1768

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Seow H et al. JAMA Network Open.

    Corresponding Author: Hsien Seow, PhD, Department of Oncology, McMaster University, 699 Concession St, Room 4-229, Hamilton, ON L8V 5C2, Canada (seowh@mcmaster.ca)

    Author Contributions: Drs Seow and Sutradhar had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Seow, Barbera, Earle, Guthrie, Isenberg, Myers, Brouwers, Sutradhar.

    Acquisition, analysis, or interpretation of data: Seow, Tanuseputro, Barbera, Earle, Guthrie, Isenberg, Juergens, Brouwers, Sutradhar.

    Drafting of the manuscript: Seow, Myers, Sutradhar.

    Critical revision of the manuscript for important intellectual content: Seow, Tanuseputro, Barbera, Earle, Guthrie, Isenberg, Juergens, Brouwers, Sutradhar.

    Statistical analysis: Seow, Guthrie, Myers, Sutradhar.

    Obtained funding: Seow, Barbera, Sutradhar.

    Administrative, technical, or material support: Seow, Tanuseputro, Isenberg.

    Supervision: Tanuseputro, Sutradhar.

    Conflict of Interest Disclosures: Dr Barbera reported receiving personal fees from Genentech outside the submitted work. Dr Juergens reported receiving personal fees from AbbVie, Amgen, EMD Serono, Fusion Pharmaceuticals, Novartis, Pfizer, Roche, and Takeda outside the submitted work and receiving grants and personal fees from AstraZeneca, Bristol-Myers Squibb, and Merck Sharp and Dohme. No other disclosures were reported.

    Funding/Support: This study was funded by grants 379009 and 383402 from the Canadian Institutes for Health Research. The study used databases maintained by ICES (formerly known as the Institute for Clinical Evaluative Sciences), Cancer Care Ontario, Canadian Institutes of Health Information, and the Ontario Association of Community Care Access Centers, which receive funding from the Ontario Ministry of Health and Long-Term Care.

    Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Disclaimer: The opinions, results, and conclusions reported in this article are those of the authors solely.

    Additional Contributions: The following people provided analytic and coordination support as paid staff: Julia Ma, MPH, McMaster University; Erin O’Leary, MSc, McMaster University; Semra Tibebu, MPH; and Amina Benmessaoud, MEngD. Lesley Moody, PhD, provided helpful comments during the data analysis and was not compensated for this work.

    References
    1.
    Bakitas  M, Lyons  KD, Hegel  MT,  et al.  Effects of a palliative care intervention on clinical outcomes in patients with advanced cancer: the Project ENABLE II randomized controlled trial.  JAMA. 2009;302(7):741-749. doi:10.1001/jama.2009.1198PubMedGoogle ScholarCrossref
    2.
    Zimmermann  C, Swami  N, Krzyzanowska  M,  et al.  Early palliative care for patients with advanced cancer: a cluster-randomised controlled trial.  Lancet. 2014;383(9930):1721-1730. doi:10.1016/S0140-6736(13)62416-2PubMedGoogle ScholarCrossref
    3.
    Temel  JS, Greer  JA, Muzikansky  A,  et al.  Early palliative care for patients with metastatic non-small-cell lung cancer.  N Engl J Med. 2010;363(8):733-742. doi:10.1056/NEJMoa1000678PubMedGoogle ScholarCrossref
    4.
    Ferrell  BR, Temel  JS, Temin  S, Smith  TJ.  Integration of palliative care into standard oncology care: ASCO clinical practice guideline update summary.  J Oncol Pract. 2017;13(2):119-121. doi:10.1200/JOP.2016.017897PubMedGoogle ScholarCrossref
    5.
    National Hospice and Palliative Care Organization.  NHPCO Facts and Figures: Hospice Care in America. National Hospice and Palliative Care Organization; 2016.
    6.
    Rabin  BA, Gaglio  B, Sanders  T,  et al.  Predicting cancer prognosis using interactive online tools: a systematic review and implications for cancer care providers.  Cancer Epidemiol Biomarkers Prev. 2013;22(10):1645-1656. doi:10.1158/1055-9965.EPI-13-0513PubMedGoogle ScholarCrossref
    7.
    Zhi  WI, Smith  TJ.  Early integration of palliative care into oncology: evidence, challenges and barriers.  Ann Palliat Med. 2015;4(3):122-131.PubMedGoogle Scholar
    8.
    Fassbender  K, Watanabe  SM.  Early palliative care and its translation into oncology practice in Canada: barriers and challenges.  Ann Palliat Med. 2015;4(3):135-149.PubMedGoogle Scholar
    9.
    Cheon  S, Agarwal  A, Popovic  M,  et al.  The accuracy of clinicians’ predictions of survival in advanced cancer: a review.  Ann Palliat Med. 2016;5(1):22-29.PubMedGoogle Scholar
    10.
    Chow  E, Harth  T, Hruby  G, Finkelstein  J, Wu  J, Danjoux  C.  How accurate are physicians’ clinical predictions of survival and the available prognostic tools in estimating survival times in terminally ill cancer patients? a systematic review.  Clin Oncol (R Coll Radiol). 2001;13(3):209-218. doi:10.1007/s001740170078PubMedGoogle Scholar
    11.
    Sutradhar  R, Rostami  M, Barbera  L.  Patient-reported symptoms improve performance of risk prediction models for emergency department visits among patients with cancer: a population-wide study in Ontario using administrative data.  J Pain Symptom Manage. 2019;58(5):745-755. doi:10.1016/j.jpainsymman.2019.07.007PubMedGoogle ScholarCrossref
    12.
    Seow  H, Barbera  L, Sutradhar  R,  et al.  Trajectory of performance status and symptom scores for patients with cancer during the last six months of life.  J Clin Oncol. 2011;29(9):1151-1158. doi:10.1200/JCO.2010.30.7173PubMedGoogle ScholarCrossref
    13.
    Maltoni  M, Caraceni  A, Brunelli  C,  et al; Steering Committee of the European Association for Palliative Care.  Prognostic factors in advanced cancer patients: evidence-based clinical recommendations: a study by the Steering Committee of the European Association for Palliative Care.  J Clin Oncol. 2005;23(25):6240-6248. doi:10.1200/JCO.2005.06.866PubMedGoogle ScholarCrossref
    14.
    Stillwell  AP, Ho  YH, Veitch  C.  Systematic review of prognostic factors related to overall survival in patients with stage IV colorectal cancer and unresectable metastases.  World J Surg. 2011;35(3):684-692. doi:10.1007/s00268-010-0891-8PubMedGoogle ScholarCrossref
    15.
    Barbera  L, Seow  H, Howell  D,  et al.  Symptom burden and performance status in a population-based cohort of ambulatory cancer patients.  Cancer. 2010;116(24):5767-5776. doi:10.1002/cncr.25681PubMedGoogle ScholarCrossref
    16.
    Guthrie  DM, Harman  LE, Barbera  L,  et al.  Quality indicator rates for seriously ill home care clients: analysis of resident assessment instrument for home care data in six Canadian provinces.  J Palliat Med. 2019;22(11):1346-1356. doi:10.1089/jpm.2019.0022PubMedGoogle ScholarCrossref
    17.
    Collins  GS, Reitsma  JB, Altman  DG, Moons  KG.  Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement.  BMJ. 2015;350:g7594. doi:10.1136/bmj.g7594PubMedGoogle ScholarCrossref
    18.
    Anderson  F, Downing  GM, Hill  J, Casorso  L, Lerch  N.  Palliative performance scale (PPS): a new tool.  J Palliat Care. 1996;12(1):5-11. doi:10.1177/082585979601200102PubMedGoogle ScholarCrossref
    19.
    Chang  VT, Hwang  SS, Feuerman  M.  Validation of the Edmonton Symptom Assessment Scale.  Cancer. 2000;88(9):2164-2171. doi:10.1002/(SICI)1097-0142(20000501)88:9<2164::AID-CNCR24>3.0.CO;2-5PubMedGoogle ScholarCrossref
    20.
    Canadian Quality Council of Ontario. Cancer System Quality Index. 2016. Accessed October 25, 2012. https://www.csqi.on.ca/
    21.
    Popovic  G, Harhara  T, Pope  A,  et al.  Patient-reported functional status in outpatients with advanced cancer: correlation with physician-reported scores and survival.  J Pain Symptom Manage. 2018;55(6):1500-1508. doi:10.1016/j.jpainsymman.2018.02.015PubMedGoogle ScholarCrossref
    22.
    Martin  L, Watanabe  S, Fainsinger  R,  et al.  Prognostic factors in patients with advanced cancer: use of the patient-generated subjective global assessment in survival prediction.  J Clin Oncol. 2010;28(28):4376-4383. doi:10.1200/JCO.2009.27.1916PubMedGoogle ScholarCrossref
    23.
    Hirdes  JP, Ljunggren  G, Morris  JN,  et al.  Reliability of the interRAI suite of assessment instruments: a 12-country study of an integrated health information system.  BMC Health Serv Res. 2008;8:277. doi:10.1186/1472-6963-8-277PubMedGoogle ScholarCrossref
    24.
    Morris  JN, Fries  BE, Steel  K,  et al.  Comprehensive clinical assessment in community setting: applicability of the MDS-HC.  J Am Geriatr Soc. 1997;45(8):1017-1024. doi:10.1111/j.1532-5415.1997.tb02975.xPubMedGoogle ScholarCrossref
    25.
    Poss  JW, Jutan  NM, Hirdes  JP,  et al.  A review of evidence on the reliability and validity of Minimum Data Set data.  Healthc Manage Forum. 2008;21(1):33-39. doi:10.1016/S0840-4704(10)60127-5PubMedGoogle ScholarCrossref
    26.
    Barbera  L, Sussman  J, Viola  R,  et al.  Factors associated with end-of-life health service use in patients dying of cancer.  Healthc Policy. 2010;5(3):e125-e143. doi:10.12927/hcpol.2013.21644PubMedGoogle Scholar
    27.
    Hirdes  JP, Frijters  DH, Teare  GF.  The MDS-CHESS scale: a new measure to predict mortality in institutionalized older people.  J Am Geriatr Soc. 2003;51(1):96-100. doi:10.1034/j.1601-5215.2002.51017.xPubMedGoogle ScholarCrossref
    28.
    Muggah  E, Graves  E, Bennett  C, Manuel  DG.  The impact of multiple chronic diseases on ambulatory care use; a population based study in Ontario, Canada.  BMC Health Serv Res. 2012;12:452. doi:10.1186/1472-6963-12-452PubMedGoogle ScholarCrossref
    29.
    Muggah  E, Graves  E, Bennett  C, Manuel  DG.  Ascertainment of chronic diseases using population health data: a comparison of health administrative data and patient self-report.  BMC Public Health. 2013;13:16. doi:10.1186/1471-2458-13-16PubMedGoogle ScholarCrossref
    30.
    Agopian  VG, Harlander-Locke  M, Zarrinpar  A,  et al.  A novel prognostic nomogram accurately predicts hepatocellular carcinoma recurrence after liver transplantation: analysis of 865 consecutive liver transplant recipients.  J Am Coll Surg. 2015;220(4):416-427. doi:10.1016/j.jamcollsurg.2014.12.025PubMedGoogle ScholarCrossref
    31.
    Kawai  K, Ishihara  S, Yamaguchi  H,  et al.  Nomograms for predicting the prognosis of stage IV colorectal cancer after curative resection: a multicenter retrospective study.  Eur J Surg Oncol. 2015;41(4):457-465. doi:10.1016/j.ejso.2015.01.026PubMedGoogle ScholarCrossref
    32.
    Harrell  F.  Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis. Springer-Verlag; 2001. doi:10.1007/978-1-4757-3462-1
    33.
    O’Brien  BA, Cohen  RJ, Ryan  A, Sengupta  S, Mills  J.  A new preoperative nomogram to predict minimal prostate cancer: accuracy and error rates compared to other tools to select patients for active surveillance.  J Urol. 2011;186(5):1811-1817. doi:10.1016/j.juro.2011.06.060PubMedGoogle ScholarCrossref
    34.
    Therneau  TM, Grambsch  PM.  Modeling Survival Data: Extending the Cox Model. Springer; 2000. doi:10.1007/978-1-4757-3294-8
    35.
    Yi  M, Meric-Bernstam  F, Kuerer  HM,  et al.  Evaluation of a breast cancer nomogram for predicting risk of ipsilateral breast tumor recurrences in patients with ductal carcinoma in situ after local excision.  J Clin Oncol. 2012;30(6):600-607. doi:10.1200/JCO.2011.36.4976PubMedGoogle ScholarCrossref
    36.
    Balachandran  VP, Gonen  M, Smith  JJ, DeMatteo  RP.  Nomograms in oncology: more than meets the eye.  Lancet Oncol. 2015;16(4):e173-e180. doi:10.1016/S1470-2045(14)71116-7PubMedGoogle ScholarCrossref
    37.
    Su  J, Barbera  L, Sutradhar  R.  Do repeated assessments of performance status improve predictions for risk of death among patients with cancer? a population-based cohort study.  Palliat Med. 2015;29(6):547-553. doi:10.1177/0269216314568231PubMedGoogle ScholarCrossref
    38.
    PROVIEW calculator. Accessed March 3, 2020. www.individualizedhealth.ca
    39.
    Wishart  GC, Bajdik  CD, Dicks  E,  et al.  PREDICT Plus: development and validation of a prognostic model for early breast cancer that includes HER2.  Br J Cancer. 2012;107(5):800-807. doi:10.1038/bjc.2012.338PubMedGoogle ScholarCrossref
    40.
    Kattan  MW, Cuzick  J, Fisher  G,  et al; Transatlantic Prostate Group.  Nomogram incorporating PSA level to predict cancer-specific survival for men with clinically localized prostate cancer managed without curative intent.  Cancer. 2008;112(1):69-74. doi:10.1002/cncr.23106PubMedGoogle ScholarCrossref
    41.
    Stephenson  AJ, Kattan  MW, Eastham  JA,  et al.  Prostate cancer-specific mortality after radical prostatectomy for patients treated in the prostate-specific antigen era.  J Clin Oncol. 2009;27(26):4300-4305. doi:10.1200/JCO.2008.18.2501PubMedGoogle ScholarCrossref
    42.
    Isenberg  SR, Crossnohere  NL, Patel  MI,  et al.  An advance care plan decision support video before major surgery: a patient- and family-centred approach.  BMJ Support Palliat Care. 2018;8(2):229-236. doi:10.1136/bmjspcare-2017-001449PubMedGoogle ScholarCrossref
    ×