[Skip to Navigation]
Sign In
Visual Abstract. Electronic Symptom Monitoring With Patient-Reported Surveys and Outcomes Among Patients With Metastatic Cancer
Electronic Symptom Monitoring With Patient-Reported Surveys and Outcomes Among Patients With Metastatic Cancer
Figure 1.  Participant Flow in the PRO-TECT Trial
Participant Flow in the PRO-TECT Trial

PRO indicates patient-reported outcome; QLQ-C30, European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire. Patient cluster size: intervention group, mean, 22.8 (SD, 12.5) [range, 2-50]; control group, mean, 23.0 (SD, 8.6) [range, 3-42]. Rural/urban practice location (based on 2010 US Census data, confirmed with practice self-designation): intervention group, 7 rural (26.9%); 19 urban (73.1%); control group, 7 rural (26.9%); 19 urban (73.1%).

Figure 2.  Score Distribution and Model-Based Mean Change From Baseline at Each Assessment Time Point for Physical Function, Symptom Control, and Health-Related Quality of Life
Score Distribution and Model-Based Mean Change From Baseline at Each Assessment Time Point for Physical Function, Symptom Control, and Health-Related Quality of Life

Scores on the European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire. PRO indicates patient-reported outcome. Positive values represent improvement. In panels A-C, P values for model-based mean change from baseline at 1-, 3-, 6-, 9-, and 12-month assessments, respectively, are as follows: panel A (physical function), P = .21, P = .02, P = .003, P = .02, and P = .68; panel B (symptom control), P = .003, P = .002, P = .02, P = .045, and P = .32; and panel C (health-related quality of life), P = .003, P = .002, P = .006, P = .03, and P = .24. In panels D-F, circles indicate means; horizontal bars, medians; box tops and bottoms, IQRs; and whiskers, 1.5× the IQRs. See eTable 5 in Supplement 2 for point estimates and confidence intervals.

Figure 3.  Proportions of Patients Completing Expected Patient-Reported Outcome Surveys, Patients Remaining in the Study, and Completed Surveys at Each Week of Participation
Proportions of Patients Completing Expected Patient-Reported Outcome Surveys, Patients Remaining in the Study, and Completed Surveys at Each Week of Participation

The number of patients shown below the x-axis is the number of patients remaining in the study in that week.

Table.  Participant Characteristics
Participant Characteristics
Supplement 2.

eTable 1. Patient-Reported Outcome (PRO) Weekly Survey Items and Alert Notification Criteria

eTable 2. Subgroup Analysis Variables

eTable 3. Roster of Participating Practices, Locations, Allocations, and Patient Accrual

eTable 4. Intracluster Correlation Coefficients for Each Patient-Reported Outcome Endpoint

eTable 5. Physical Function, Symptom Control, and Health-Related Quality of Life Mean Score Estimates and Differences in Mean Changes From Baseline Between Arms at Each Visit

eTable 6. Proportion of Patients in the Patient-Reported Outcome (PRO) Intervention Arm and Control Arm With 5-Point and 10-Point Changes in Physical Function, Symptom Control, and Health-Related Quality of Life

eTable 7. Subgroup Analysis of Physical Function

eTable 8. Individual Symptom Scale Mean Score Estimates and Differences Between Groups in Mean Change From Baseline at Each Measured Timepoint, From the EORTC QLQ-C30 Questionnaire

eTable 9. Completion Rates by Study Week

eTable 10. Alerts Triggered by the Weekly Survey System to the Care Team, by Symptom

eFigure 1. Example Patient-Level Educational Materials for Home Symptom Self-management

eFigure 2. Example Clinician-Level Educational Materials for Symptom Management

eFigure 3. Example Actual Clinician Report Showing Longitudinal Trajectory of Patient-Reported Outcomes (for Visualizing/Printing at Clinic Visit)

eFigure 4. Responder Analysis

eFigure 5. Preplanned Subgroup Analysis of Physical Function, as Measured by the EORTC QLQ-C30

eFigure 6. Responder Sensitivity Analysis

eFigure 7. Mean Changes From Baseline at Each Visit for Symptom Scales

1.
Cleeland  CS, Zhao  F, Chang  VT,  et al.  The symptom burden of cancer: evidence for a core set of cancer-related and treatment-related symptoms from the Eastern Cooperative Oncology Group Symptom Outcomes and Practice Patterns study.   Cancer. 2013;119(24):4333-4340. doi:10.1002/cncr.28376PubMedGoogle ScholarCrossref
2.
Reilly  CM, Bruner  DW, Mitchell  SA,  et al.  A literature synthesis of symptom prevalence and severity in persons receiving active cancer treatment.   Support Care Cancer. 2013;21(6):1525-1550. doi:10.1007/s00520-012-1688-0PubMedGoogle ScholarCrossref
3.
Panattoni  L, Fedorenko  C, Greenwood-Hickman  MA,  et al.  Characterizing potentially preventable cancer- and chronic disease-related emergency department use in the year after treatment initiation: a regional study.   J Oncol Pract. 2018;14(3):e176-e185. doi:10.1200/JOP.2017.028191PubMedGoogle ScholarCrossref
4.
Henry  DH, Viswanathan  HN, Elkin  EP, Traina  S, Wade  S, Cella  D.  Symptoms and treatment burden associated with cancer treatment: results from a cross-sectional national survey in the US.   Support Care Cancer. 2008;16(7):791-801. doi:10.1007/s00520-007-0380-2PubMedGoogle ScholarCrossref
5.
Di Maio  M, Gallo  C, Leighl  NB,  et al.  Symptomatic toxicities experienced during anticancer treatment: agreement between patient and physician reporting in three randomized trials.   J Clin Oncol. 2015;33(8):910-915. doi:10.1200/JCO.2014.57.9334PubMedGoogle ScholarCrossref
6.
Laugsand  EA, Sprangers  MAG, Bjordal  K, Skorpen  F, Kaasa  S, Klepstad  P.  Health care providers underestimate symptom intensities of cancer patients: a multicenter European study.   Health Qual Life Outcomes. 2010;8:104. doi:10.1186/1477-7525-8-104PubMedGoogle ScholarCrossref
7.
Basch  E, Iasonos  A, McDonough  T,  et al.  Patient versus clinician symptom reporting using the National Cancer Institute Common Terminology Criteria for Adverse Events: results of a questionnaire-based study.   Lancet Oncol. 2006;7(11):903-909. doi:10.1016/S1470-2045(06)70910-XPubMedGoogle ScholarCrossref
8.
Warrington  L, Absolom  K, Conner  M,  et al.  Electronic systems for patients to report and manage side effects of cancer treatment: systematic review.   J Med Internet Res. 2019;21(1):e10875. doi:10.2196/10875PubMedGoogle ScholarCrossref
9.
Kotronoulas  G, Kearney  N, Maguire  R,  et al.  What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? a systematic review of controlled trials.   J Clin Oncol. 2014;32(14):1480-1501. doi:10.1200/JCO.2013.53.5948PubMedGoogle ScholarCrossref
10.
Basch  E, Deal  AM, Kris  MG,  et al.  Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial.   J Clin Oncol. 2016;34(6):557-565. doi:10.1200/JCO.2015.63.0830PubMedGoogle ScholarCrossref
11.
Basch  E, Deal  AM, Dueck  AC,  et al.  Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment.   JAMA. 2017;318(2):197-198. doi:10.1001/jama.2017.7156PubMedGoogle ScholarCrossref
12.
Denis  F, Basch  E, Septans  AL,  et al.  Two-year survival comparing web-based symptom monitoring vs routine surveillance following treatment for lung cancer.   JAMA. 2019;321(3):306-307. doi:10.1001/jama.2018.18085PubMedGoogle ScholarCrossref
13.
Barbera  L, Sutradhar  R, Seow  H,  et al.  Impact of standardized Edmonton Symptom Assessment System use on emergency department visits and hospitalization: results of a population-based retrospective matched cohort analysis.   JCO Oncol Pract. 2020;16(9):e958-e965. doi:10.1200/JOP.19.00660PubMedGoogle ScholarCrossref
14.
Mir  O, Ferrua  M, Fourcade  A,  et al.  Digital remote monitoring plus usual care versus usual care in patients treated with oral anticancer agents: the randomized phase 3 CAPRI trial.   Nat Med. Published online April 25, 2022. doi:10.1038/s41591-022-01788-1PubMedGoogle ScholarCrossref
15.
Schmidt  T, Valuck  T, Perkins  B,  et al.  Improving patient-reported measures in oncology: a payer call to action.   J Manag Care Spec Pharm. 2021;27(1):118-126. doi:10.18553/jmcp.2020.20313PubMedGoogle ScholarCrossref
16.
Samuel  CA, Schaal  J, Robertson  L,  et al.  Racial differences in symptom management experiences during breast cancer treatment.   Support Care Cancer. 2018;26(5):1425-1435. doi:10.1007/s00520-017-3965-4PubMedGoogle ScholarCrossref
17.
Patient-Reported Outcomes Core of the University of North Carolina. Accessed May 27, 2022. https://pro.unc.edu/
18.
Basch  E, Stover  AM, Schrag  D,  et al.  Clinical utility and user perceptions of a digital system for electronic patient-reported symptom monitoring during routine cancer care: findings from the PRO-TECT trial.   JCO Clin Cancer Inform. 2020;4:947-957. doi:10.1200/CCI.20.00081PubMedGoogle ScholarCrossref
19.
Schoen  MW, Basch  E, Hudson  LL,  et al.  Software for administering the National Cancer Institute’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events: usability study.   JMIR Hum Factors. 2018;5(3):e10070. doi:10.2196/10070PubMedGoogle ScholarCrossref
20.
Basch  E, Barbera  L, Kerrigan  CL, Velikova  G.  Implementation of patient-reported outcomes in routine medical care.   Am Soc Clin Oncol Educ Book. 2018;38:122-134. doi:10.1200/EDBK_200383PubMedGoogle ScholarCrossref
21.
National Cancer Institute. Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events. Accessed May 27, 2022. https://healthcaredelivery.cancer.gov/pro-ctcae
22.
Dueck  AC, Mendoza  TR, Mitchell  SA,  et al; National Cancer Institute PRO-CTCAE Study Group.  Validity and reliability of the US National Cancer Institute’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE).   JAMA Oncol. 2015;1(8):1051-1059. doi:10.1001/jamaoncol.2015.2639PubMedGoogle ScholarCrossref
23.
Al-Rashdan  A, Sutradhar  R, Nazeri-Rad  N, Yao  C, Barbera  L.  Comparing the ability of physician-reported versus patient-reported performance status to predict survival in a population-based cohort of newly diagnosed cancer patients.  2021;33(7):476-482.  Clin Oncol (R Coll Radiol). doi:10.1016/j.clon.2021.01.008PubMedGoogle ScholarCrossref
24.
de Souza  JA, Yap  BJ, Wroblewski  K,  et al.  Measuring financial toxicity as a clinically relevant patient-reported outcome: the validation of the Comprehensive Score for Financial Toxicity (COST).   Cancer. 2017;123(3):476-484. doi:10.1002/cncr.30369PubMedGoogle ScholarCrossref
25.
Aaronson  NK, Ahmedzai  S, Bergman  B,  et al.  The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology.   J Natl Cancer Inst. 1993;85(5):365-376. doi:10.1093/jnci/85.5.365PubMedGoogle ScholarCrossref
26.
Fayers  P, Bottomley  A; EORTC Quality of Life Group; Quality of Life Unit; European Organisation for Research and Treatment of Cancer.  Quality of life research within the EORTC—the EORTC QLQ-C30.   Eur J Cancer. 2002;38(suppl 4):S125-S133. doi:10.1016/S0959-8049(01)00448-8PubMedGoogle ScholarCrossref
27.
Gundy  CM, Fayers  PM, Groenvold  M,  et al.  Comparing higher order models for the EORTC QLQ-C30.   Qual Life Res. 2012;21(9):1607-1617. doi:10.1007/s11136-011-0082-6PubMedGoogle ScholarCrossref
28.
Fayers  PM, Aaronson  NK, Bjordal  K, Groenvold  M, Curran  D, Bottomley  A; EORTC Quality of Life Group.  The EORTC QLQ-C30 Scoring Manual (3rd Edition). European Organisation for Research and Treatment of Cancer; 2001.
29.
Cocks  K, King  MT, Velikova  G,  et al.  Evidence-based guidelines for interpreting change scores for the European Organisation for the Research and Treatment of Cancer Quality of Life Questionnaire Core 30.   Eur J Cancer. 2012;48(11):1713-1721. doi:10.1016/j.ejca.2012.02.059PubMedGoogle ScholarCrossref
30.
Coens  C, Pe  M, Dueck  AC,  et al; Setting International Standards in Analyzing Patient-Reported Outcomes and Quality of Life Endpoints Data Consortium.  International standards for the analysis of quality-of-life and patient-reported outcome endpoints in cancer randomised controlled trials: recommendations of the SISAQOL Consortium.   Lancet Oncol. 2020;21(2):e83-e96. doi:10.1016/S1470-2045(19)30790-9PubMedGoogle ScholarCrossref
31.
Osoba  D, Rodrigues  G, Myles  J, Zee  B, Pater  J.  Interpreting the significance of changes in health-related quality-of-life scores.   J Clin Oncol. 1998;16(1):139-144. doi:10.1200/JCO.1998.16.1.139PubMedGoogle ScholarCrossref
32.
Basch  E, Abernethy  AP.  Encouraging clinicians to incorporate longitudinal patient-reported symptoms in routine clinical practice.   J Oncol Pract. 2011;7(1):23-25. doi:10.1200/JOP.2010.000186PubMedGoogle ScholarCrossref
33.
Lizée  T, Basch  E, Trémolières  P,  et al.  Cost-effectiveness of web-based patient-reported outcome surveillance in patients with lung cancer.   J Thorac Oncol. 2019;14(6):1012-1020. doi:10.1016/j.jtho.2019.02.005PubMedGoogle ScholarCrossref
34.
Carlotto  A, Hogsett  VL, Maiorini  EM, Razulis  JG, Sonis  ST.  The economic burden of toxicities associated with cancer treatment: review of the literature and analysis of nausea and vomiting, diarrhoea, oral mucositis and fatigue.   Pharmacoeconomics. 2013;31(9):753-766. doi:10.1007/s40273-013-0081-2PubMedGoogle ScholarCrossref
35.
Centers for Medicare & Medicaid Services. Informal request for information on proposed Oncology Care First model. Accessed May 27, 2022. https://innovation.cms.gov/Files/x/ocf-informalrfi.pdf
36.
National Academies of Sciences, Engineering, and Medicine. Establishing Effective Patient Navigation Programs in Oncology: Proceedings of a Workshop. National Academies Press; 2018.
37.
Stover  AM, Henson  S, Jansen  J,  et al.  Demographic and symptom differences in PRO-TECT trial (AFT-39) cancer patients electing to complete weekly home patient-reported outcome measures (PROMs) via an automated phone call vs. email: implications for implementing PROs into routine care.   Qual Life Res. 2019;28(suppl 1):S1.Google Scholar
Original Investigation
June 5, 2022

Effect of Electronic Symptom Monitoring on Patient-Reported Outcomes Among Patients With Metastatic Cancer: A Randomized Clinical Trial

Author Affiliations
  • 1Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill
  • 2Memorial Sloan Kettering Cancer Center, New York, New York
  • 3Mayo Clinic, Scottsdale, Arizona
  • 4Division of Hematology, Mayo Clinic, Rochester, Minnesota
  • 5Duke Cancer Institute, Duke University School of Medicine, Durham, North Carolina
  • 6Johns Hopkins Schools of Medicine and Public Health, Baltimore, Maryland
  • 7Emory University, Atlanta, Georgia
  • 8Feinberg School of Medicine, Northwestern University, Chicago, Illinois
  • 9Department of Medical Oncology, Mayo Clinic, Rochester, Minnesota
  • 10Gemini Group, Ann Arbor, Michigan
  • 11Patient and Partners, Madison, Connecticut
  • 12College of Nursing, Michigan State University, East Lansing
  • 13American Society of Clinical Oncology, Alexandria, Virginia
  • 14Christiana Care Health Services, Wilmington, Delaware
  • 15The Cancer Research Center, HealthPartners/Park Nicollet, Minneapolis, Minnesota
  • 16Brigham and Women’s Hospital, Boston, Massachusetts
JAMA. 2022;327(24):2413-2422. doi:10.1001/jama.2022.9265
Visual Abstract. Electronic Symptom Monitoring With Patient-Reported Surveys and Outcomes Among Patients With Metastatic Cancer
Electronic Symptom Monitoring With Patient-Reported Surveys and Outcomes Among Patients With Metastatic Cancer
Key Points

Question  In patients undergoing cancer treatment, does electronic symptom monitoring improve quality-of-life (QOL) outcomes?

Findings  In this cluster randomized trial that included 52 oncology practices and 1191 patients, an intervention consisting of patient-reported surveys completed electronically each week to monitor symptoms, compared with usual care, resulted in statistically significant improvements in QOL outcomes at 3 months (measured as difference in change from baseline using the European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire; range, 0-100 points) for physical function (mean difference, 2.47 points; minimum clinically important difference [MCID], 2-7), symptom control (mean difference, 2.56 points; no established MCID), and health-related QOL (mean difference, 2.43; no established MCID).

Meaning  Among adults receiving treatment for metastatic cancer, weekly electronic symptom monitoring resulted in statistically significant mean improvements of approximately 2.5 points on a 0- to 100-point scale in QOL outcomes at 3 months; the findings should be interpreted provisionally, pending results of the study’s primary outcome of survival.

Abstract

Importance  Electronic systems that facilitate patient-reported outcome (PRO) surveys for patients with cancer may detect symptoms early and prompt clinicians to intervene.

Objective  To evaluate whether electronic symptom monitoring during cancer treatment confers benefits on quality-of-life outcomes.

Design, Setting, and Participants  Report of secondary outcomes from the PRO-TECT (Alliance AFT-39) cluster randomized trial in 52 US community oncology practices randomized to electronic symptom monitoring with PRO surveys or usual care. Between October 2017 and March 2020, 1191 adults being treated for metastatic cancer were enrolled, with last follow-up on May 17, 2021.

Interventions  In the PRO group, participants (n = 593) were asked to complete weekly surveys via an internet-based or automated telephone system for up to 1 year. Severe or worsening symptoms triggered care team alerts. The control group (n = 598) received usual care.

Main Outcomes and Measures  The 3 prespecified secondary outcomes were physical function, symptom control, and health-related quality of life (HRQOL) at 3 months, measured by the European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire (QLQ-C30; range, 0-100 points; minimum clinically important difference [MCID], 2-7 for physical function; no MCID defined for symptom control or HRQOL). Results on the primary outcome, overall survival, are not yet available.

Results  Among 52 practices, 1191 patients were included (mean age, 62.2 years; 694 [58.3%] women); 1066 (89.5%) completed 3-month follow-up. Compared with usual care, mean changes on the QLQ-C30 from baseline to 3 months were significantly improved in the PRO group for physical function (PRO, from 74.27 to 75.81 points; control, from 73.54 to 72.61 points; mean difference, 2.47 [95% CI, 0.41-4.53]; P = .02), symptom control (PRO, from 77.67 to 80.03 points; control, from 76.75 to 76.55 points; mean difference, 2.56 [95% CI, 0.95-4.17]; P = .002), and HRQOL (PRO, from 78.11 to 80.03 points; control, from 77.00 to 76.50 points; mean difference, 2.43 [95% CI, 0.90-3.96]; P = .002). Patients in the PRO group had significantly greater odds of experiencing clinically meaningful benefits vs usual care for physical function (7.7% more with improvements of ≥5 points and 6.1% fewer with worsening of ≥5 points; odds ratio [OR], 1.35 [95% CI, 1.08-1.70]; P = .009), symptom control (8.6% and 7.5%, respectively; OR, 1.50 [95% CI, 1.15-1.95]; P = .003), and HRQOL (8.5% and 4.9%, respectively; OR, 1.41 [95% CI, 1.10-1.81]; P = .006).

Conclusions and Relevance  In this report of secondary outcomes from a randomized clinical trial of adults receiving cancer treatment, use of weekly electronic PRO surveys to monitor symptoms, compared with usual care, resulted in statistically significant improvements in physical function, symptom control, and HRQOL at 3 months, with mean improvements of approximately 2.5 points on a 0- to 100-point scale. These findings should be interpreted provisionally pending results of the primary outcome of overall survival.

Trial Registration  ClinicalTrials.gov Identifier: NCT03249090

Introduction

Symptom management is an essential component of treating patients with cancer. Patients with cancer frequently experience symptoms related to disease and treatment,1,2 which can contribute to distress and complications.3,4 However, symptoms during treatment of cancer are frequently undetected and untreated.5-7 Interventions that detect and treat symptoms may improve outcomes in people undergoing treatment for cancer.

Symptom monitoring using electronic systems that facilitate patient-reported outcome (PRO) surveys via the internet, mobile applications, or automated telephone interfaces have been shown to be feasible for identifying symptoms that can be treated by clinicians.8 Prior studies have reported improved outcomes for physical function, symptom control, health-related quality of life (HRQOL), hospitalizations, and survival when such electronic systems are used by patients receiving cancer treatment.8-14 Integrating symptom monitoring into routine treatment of cancer may improve outcomes with an opportunity for reimbursement by payers.15

Based on findings from a prior single-center study,10 a multicenter randomized clinical trial, the PRO-TECT trial (Alliance AFT-39), was conducted across 52 US-based community oncology practices to determine whether electronic monitoring of symptoms by patients undergoing treatment for metastatic cancer improves survival, QOL outcomes, and other outcomes compared with usual care. In this report, the QOL secondary outcomes and selected exploratory outcomes are presented.10 Analysis of the trial’s primary outcome, overall survival, requires national administrative data that are not yet available.

Methods
Patients

This was a multicenter cluster randomized trial evaluating electronic symptom monitoring with PRO surveys compared with a usual care control group. All participants signed informed consent. The trial protocol and consent form were approved by central and local institutional review boards. The statistical analysis plan and trial protocol were finalized prior to data analysis and are available in Supplement 1.

Community oncology practices in the US national network of the Alliance for Clinical Trials in Oncology (http://www.allianceforclinicaltrialsinoncology.org) were invited to participate. Practices were asked to consecutively approach and enroll up to 50 adults with metastatic cancer of any type receiving treatment with chemotherapy, targeted oral therapy, and/or immunotherapy if they understood English, Spanish, or Mandarin. Patients with indolent lymphoma or acute leukemia or who were receiving hormonal monotherapy were excluded. Full eligibility is listed in the trial protocol (Supplement 1). Race and ethnicity information was obtained from patients using fixed categories to determine whether effects of the intervention differed by race or ethnicity.16

Randomization

Participating practices were stratified by rural/urban designation using US Census criteria and randomized using permuted blocks with block sizes of 2 or 4. Randomization lists were computer generated based on random numbers and concealed until sites were randomized. A cluster randomized design was used to avoid influencing symptom management procedures in people randomized to usual care.

Intervention

Practices received online access to standardized educational materials for managing symptoms, including patient-level and clinician-level versions (eFigures 1 and 2 in Supplement 2). Practices randomized to PROs received access to the electronic survey system based at the University of North Carolina’s PRO-Core facility.17

The electronic survey system was developed based on prior publications.18-20 The system included a survey with questions from the National Cancer Institute’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) validated item library,21,22 selected based on prevalence across different types of advanced cancers,2 asking about symptoms of pain, nausea, vomiting, constipation, diarrhea, dyspnea, insomnia, and depression, as well as questions about oral intake (eating/drinking), performance status (patient-reported Eastern Cooperative Oncology Group [ECOG] criteria),23 falls, and financial challenges24 (eTable 1 in Supplement 2). Surveys were available in English, Spanish, or Mandarin. Patients could complete surveys via the internet or an automated telephone system.

Patients at practices randomized to the PRO intervention were asked to complete surveys weekly for 1 year or until they discontinued all cancer treatment. On a day of the week and time of day selected by each patient, an email or automated call prompted survey completion. If a patient did not complete the survey within 24 hours, they received a reminder prompt. If they did not complete the survey after 72 hours, they received a “backup” call from a clinic staff member reminding them to self-report or offering to administer the survey orally. Family members, caregivers, or staff were permitted to assist patients completing surveys.

Whenever a PRO-CTCAE score reached a prespecified level of magnitude or worsening compared with the prior survey (eTable 1 in Supplement 2), the patient received an email with a link to patient-level educational materials about their self-management of that symptom (eFigure 1 in Supplement 2). In addition, an email-based alert notification was sent to a designated administrative staff member at the patient’s oncology practice who routed it to a clinical nurse responsible for that patient; notifications included PRO scores and clinician-level educational materials for managing the alert symptom(s) (eFigure 2 in Supplement 2). Alerts were also sent for ECOG scores greater than 2 (range, 0-5; 4 = worst function; 5 = death), ECOG scores that worsened by 2 or more points, falls (“yes”), and financial distress scores greater than 2 (“quite a bit” or “very much”; range, “not at all” to “very much”). Reports showing the trajectory of PROs could be visualized on screen or printed for clinicians (eFigure 3 in Supplement 2). Responses to the alerts by nurses or oncologists were not dictated by the study, were left to the discretion of the clinicians, and were documented on a standardized form. The electronic system was not integrated into electronic medical record systems.

Outcomes

The initial study protocol, approved on July 27, 2017, included co–primary outcomes of physical function at 3 months and overall survival. On December 5, 2020, prior to data analysis, an amendment to the protocol changed the primary outcome to overall survival as the single primary outcome in order to increase statistical power for this outcome and because it was determined that there was more than adequate power for the physical function outcome. At this time, physical function was changed to a key secondary end point. Additional secondary outcomes included symptom control and HRQOL. Emergency department visits and duration of chemotherapy were included in a list of secondary outcomes in the initial protocol but were clarified as exploratory outcomes on April 1, 2021, prior to data analysis. The statistical analysis plan specified month 3 as the principal analysis time point for secondary outcomes, with additional outcomes collected at months 1, 6, 9, and 12.

Physical function, symptom control, and HRQOL outcomes were obtained from patients during clinic visits at baseline and months 1, 3, 6, 9, and 12 of participation via the European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire (QLQ-C30).25 The QLQ-C30 is a widely used 30-item questionnaire with established measurement properties.26 Physical function was assessed using 5 items, which generated a single score (symptom burden/control was assessed as a composite of 8 QLQ-C30 symptom scale scores27; and overall HRQOL was assessed by combining function and symptom scale scores.26,28 Each outcome was evaluated using a 0- to 100-point scale (higher scores are better). For the physical function outcome, a small minimum clinically important difference (MCID) has been defined as 2 to 7, but no MCID has been defined for symptom control or HRQOL as scored in this trial.29

Prespecified exploratory outcomes included emergency department visits and duration of chemotherapy administration, which are not reported in this article, and individual symptom items from the QLQ-C30. In the PRO group only, feedback was solicited from participants and nurses about acceptability of and satisfaction with the PRO system, after 3 and 6 months of participation, respectively.10 A post hoc analysis examined the frequency of and reasons for alert notifications as well as clinicians’ responses to them.

Patients in both groups received $75 for their time and effort related to completing a research demographics form and the QLQ-C30 at enrollment and 3 months. Data for overall survival will be obtained from administrative databases and were not available at the time this report was prepared.

Sample Size Calculation

The initial protocol included a sample size of 1000, which was increased to 1200 in an institutional review board amendment on January 17, 2019, when additional funding became available to increase the number of participants who were Black or African American, yielding 90% power for the primary outcome of overall survival, when physical function was still a co–primary outcome, for a hazard ratio of 0.76 (based on prior research10) using a 2-sided α = .05 log-rank test with 576 observed events, with an intracluster correlation coefficient of 0.001 (based on prior trials), assuming dropout of 150 patients in the first 2.5 years. When physical function became a key secondary end point on December 5, 2020, 1200 patients at 50 to 55 sites provided greater than 90% power to detect a 0.37-SD difference between randomization groups based on prior research,10 assuming an intracluster correlation coefficient of 0.055 and assuming that 85% of patients provided data at 3 months.

Statistical Analysis

Mean change from baseline in physical function, symptom control, and HRQOL were compared between groups at each visit using a linear combination of parameters from a general linear mixed model. Each model included all available data from all time points from all patients according to their randomization group. Fixed effects included study group, time point, cancer type, and group-by-visit interaction. A random practice intercept term was included to account for clustering by practice. Repeated observations by patient were modeled using compound symmetric correlation structure over time. General linear mixed modeling produces unbiased estimates under an assumption of missing-at-random data including handling patient dropout due to death or other causes.30

In a responder analysis, patients who completed the QLQ-C30 at baseline and each follow-up time point were categorized as improved on each outcome if their score increased by at least 5 points from baseline; worse if their score decreased by at least 5 points; and otherwise as stable. A 5-point change was selected based on prior research.31 In a sensitivity analysis, results were repeated using a 10-point change to define improvement or worsening. The proportion of patients with improvement, stability, or worsening was compared between groups at each time point using a cumulative logistic regression model with fixed effects for group and cancer type and a random practice intercept term to account for clustering by practice. Minimum clinically important differences were not established for the differences in response rates between groups for the measured outcomes. The primary time point for mean comparisons and responder analyses was prespecified as month 3, with additional outcomes collected at months 1, 6, 9, and 12. Preplanned subgroup analysis of physical function used the same mixed-model approach described above to compare mean changes from baseline at month 3 between groups and within subgroups. An additional mixed model was performed for interaction testing (eTable 2 in Supplement 2). Patient and nurse feedback items were analyzed descriptively. Completion of weekly PRO surveys was defined as the proportion of patients who completed an expected survey at each weekly time point.

Statistical testing was 2-sided, with P < .05 considered statistically significant, and was carried out in SAS version 9.4 (SAS Institute Inc). Because of the potential for type I error, findings for the secondary end points presented in this article should be considered exploratory.

Results

On March 23, 2020, the study chair made the decision to discontinue enrollment, with 1191 (99.3%) of 1200 planned enrollees, because practice site personnel were no longer able to recruit patients during the COVID-19 pandemic.

This trial included 52 community oncology practice sites across 25 states, randomized 1:1 to PRO or control, with 14 (26.9%) designated as rural (7 per group) (Figure 1; eTable 3 in Supplement 2). The post hoc intracluster correlation coefficient for physical function was 0.020 and varied between less than 0.001 and 0.039 for other outcomes (eTable 4 in Supplement 2). Each practice site was discrete. No staff worked at more than 1 site. Between October 31, 2017, and March 23, 2020, 1444 patients were approached and 1191 were enrolled (593 to the PRO group and 598 to the control group). Participants had a median age of 63 years (range, 28-93 years); 694 (58.3%) were female; 925 (79.5%) were White; 317 (26.6%) were recruited from a rural practice location; 468 (39.4%) had a high school education or less; and 201 (16.9%) had limited internet experience (Table). At 3 months, 1098 (97.3%) of 1191 participants remained in the study and 1066 (97.1%) of those 1098 participants had completed the QLQ-C30 (Figure 1).

Secondary Quality-of-Life Outcomes

At 3 months, the mean change from baseline on the QLQ-C30 was significantly better for the PRO group than for the control group for physical function (PRO group, from 74.27 to 75.81 points; control group, from 73.54 to 72.61 points; mean difference, 2.47 [95% CI, 0.41-4.53] points; P = .02), symptom control (PRO group, from 77.67 to 80.03 points; control group, from 76.75 to 76.55 points; mean difference, 2.56 [95% CI, 0.95-4.17] points; P = .002), and HRQOL (PRO group, from 78.11 to 80.03 points; control group, from 77.00 to 76.50 points; mean difference, 2.43 [95% CI, 0.90-3.96] points; P = .002) (Figure 2; eTable 5 in Supplement 2).

In the responder analysis, the proportion of patients with clinically meaningful (≥5-point) changes in outcomes at 3-month follow-up was significantly greater with PRO, compared with usual care, for the outcome of physical function, with 13.8% more patients in the PRO group experiencing physical function benefits (either a higher rate of improvement or a lower rate of decline) (odds ratio [OR], 1.35 [95% CI, 1.08-1.70]; P = .009), including 7.7% more with improvements (227/522 [43.5%] in the PRO group vs 195/544 [35.8%] in the control group), and 6.1% fewer with worsening (162/522 [31.0%] vs 202/544 [37.1%], respectively). There were 16.1% more patients in the PRO group with symptom control benefits compared with the control group (OR, 1.50 [95% CI, 1.15-1.95]; P = .003), including 8.6% more with improvements (195/517 [37.7%] vs 158/543 [29.1%], respectively) and 7.5% fewer with worsening (123/517 [23.8%] vs 170/543 [31.3%], respectively). There were 13.4% more patients in the PRO group with HRQOL benefits compared with the control group (OR, 1.41 [95% CI, 1.10-1.81]; P = .006), including 8.5% more patients with improvements (191/517 [36.9%] vs 154/543 [28.4%], respectively) and 4.9% fewer with worsening (127/517 [24.6%] vs 160/543 [29.5%], respectively) (eTable 6 and eFigure 4 in Supplement 2).

Analyses at Other Time Points and Subgroup Analyses

Mean differences remained statistically significant at months 6 and 9 but not at month 12 (Figure 2; eTable 5 in Supplement 2). Similarly, significant benefits in the proportion of patients with clinically meaningful changes persisted through month 9, but there was no statistically significant difference by month 12. At 12-month follow-up, 508 (42.7%) of 1191 patients were no longer participating in study follow-up (Figure 1; eTable 6 in Supplement 2). In prespecified subgroup analyses, no interaction tests were statistically significant (eTable 7 and eFigure 5 in Supplement 2).

Sensitivity Analysis

Results were statistically significant in a responder analysis of the proportion of patients with 10-point changes in outcomes at 3-month follow-up in the PRO group compared with usual care. There were 12.2% more PRO group patients who experienced physical function benefits compared with the control group (either a higher rate of improvement or a lower rate of decline) (OR, 1.43 [95% CI, 1.10-1.84]; P = .007), including 5.9% more patients with improvements (135/522 [25.9%] in the PRO group vs 109/544 [20.0%] in the control group) and 6.3% fewer patients with worsening (93/522 [17.8%] vs 131/544 [24.1%], respectively). There were 11.4% more patients who experienced symptom control benefits in the PRO group than in the control group (OR, 1.51 [95% CI, 1.14-2.00]; P = .004), including 4.2% more with improvements (118/517 [22.8%] vs 101/543 [18.6%], respectively) and 7.2% fewer with worsening (68/517 [13.2%] vs 111/543 [20.4%], respectively), and 13.1% more patients who experienced HRQOL benefits in the PRO group relative to the control group (OR, 1.69 [95% CI, 1.24-2.31]; P = .004), including 5.4% more with improvements (103/517 [19.9%] vs 79/543 [14.5%], respectively) and 7.7% fewer with worsening (62/517 [12.0%] vs 107/543 [19.7%], respectively) (eTable 6 and eFigure 6 in Supplement 2).

Exploratory Outcomes
Symptom Trajectories

In an analysis of individual symptom trajectories, significant mean differences in favor of the PRO group were observed at all time points for fatigue and at multiple time points for nausea, insomnia, appetite loss, and diarrhea but not for pain, dyspnea, or constipation (eTable 8 and eFigure 7 in Supplement 2).

Intervention Adherence and Feedback

Among PRO group patients, 20 565 (91.5%) of 22 486 expected weekly PRO surveys were completed, without substantial reduction in PRO completion rates over time (Figure 3 and eTable 9 in Supplement 2). “Backup” reminder calls by staff for patients who did not complete the electronic surveys when prompted were required for 3120 (13.9%) of 22 486 PRO surveys. A total of 18 973 (84.4%) surveys were completed by patients themselves, 846 (3.8%) by caregivers, and 606 (2.7%) by staff. A post hoc analysis found that among 20 565 completed surveys, there were 6979 (33.9%) alert notifications to nurses, most commonly for pain, diminished performance status, and diarrhea (eTable 10 in Supplement 2). Among these notifications, 4122 (59.1%) of 6979 resulted in immediate nursing actions, including telephone discussion with the patient or caregiver (n = 3097 [75.1%]), advice for self-management at home (n = 944 [22.9%]), and supportive medications prescribed or modified (n = 868 [21.1%]).

In feedback from patients in the PRO group at 3-month follow-up, 387 (76.8%) of 504 noted that electronic symptom monitoring made them feel in greater control of their care, 365 (72.4%) of 504 that it improved discussions with the care team, 359 (91.6%) of 392 that it was relevant to their care, and 452 (89.5%) of 505 that they would recommend it to other patients. Among nurses after 6 months of participation, 49 (84.5%) of 58 indicated that symptom monitoring information improved the quality of discussions with patients, 49 (84.5%) of 58 that it increased efficiency of discussions, and 46 (79.3%) of 58 that it was helpful for electronic medical record documentation. While 44 (75.9%) of 58 nurses found alerts helpful, 28 (49.1%) of 57 felt that there were too many alerts.

Dropout rates were greater in the PRO group (41/593 [6.9%]) vs control (7/598 [1.2%]). The higher dropout rates in the PRO group were attributed to survey fatigue by 9 (1.5%) of 593 in the PRO group and 3 (0.5%) of 598 in the control group.

Discussion

Among patients receiving treatment at US community oncology practices, electronic symptom monitoring with PROs improved patient-reported physical function, symptom control, and HRQOL at 3 months based on small but statistically significant mean differences between groups and based on statistically significant differences in the proportions of patients meeting the criteria for clinically meaningful benefits. Although effect sizes were smaller than in a prior single-center trial, the statistically significant findings are consistent with the prior single-center trial10,11 and are consistent with the findings of other prior studies.8-14

Symptoms are common and debilitating for patients with cancer1,2 and are often unmeasured and unrecognized between visits.5-7 Identifying symptoms early via PROs and alerting clinicians to their presence facilitated interventions to prevent subsequent symptom worsening or complications.3 This trial also demonstrated that high rates of patient survey completion can be attained in routine clinical practice—even when patients are ill and have limited prior technology experience. The intervention in this trial used best practices for implementation and focused on usability, clinical follow-up, and integration of the intervention into existing clinical practice.18,19,32 In a prior analysis, there was no significant correlation between completion of PRO surveys and prior computer experience.18

Despite prior evidence of the benefits and feasibility of electronic symptom monitoring with PROs, there has not been widespread adoption in treatment for cancer.31 Implementation requires technology, patient engagement, staff effort, and modification of information flow. A prior analysis of user feedback from this trial found high levels of enthusiasm from patients and clinicians.18 However, nurses reported that alerts could be burdensome unless there is dedicated time to address the alerts. Despite cost savings associated with PRO monitoring for insurance companies,13,33,34 currently funding is not available to sustain additional costs to clinics of technology and staffing in the US. The Centers for Medicare & Medicaid Services has suggested including PROs in a proposed oncology payment model.35 Expansion of current billing codes for remote monitoring to include PROs in oncology would facilitate greater use of the intervention. Future work could delineate the relationship of PRO monitoring with patient navigation.36 Work is needed to refine algorithms for communicating alerts to assess whether the number of notifications can be safely reduced. Among patients randomized to PRO, 215 (36.3%) of 593 chose automated telephone rather than internet for completing surveys, a choice previously associated with older age and lower education.37 This suggests that noninternet interfaces may increase participation and equity.

This trial demonstrated the feasibility, clinical utility, and clinical benefits of administering questions from the National Cancer Institute’s PRO-CTCAE item library in routine clinical practice. The PRO-CTCAE library was initially validated in a routine care setting21 yet has largely been used in clinical research. This trial demonstrated that administering the PRO-CTCAE questions for symptom monitoring in routine therapy was beneficial and feasible for people with cancer.

However, the following should also be considered. First, the effect size was smaller than demonstrated in a prior single-center trial.10 Second, the effect size diminished over time, which may have reflected reduced benefits after a year of participation. Third, dropout rates were higher in the intervention group (6.9% vs 1.2%), suggesting that some participants may have found the intervention burdensome.

Limitations

This study has several limitations. First, the primary outcome of overall survival was not available at the time of this report. Second, benefits were not attained for all patients or all symptoms. This is likely related to heterogeneity of included patients. In the tests for interactions, some subgroups had small sample sizes, limiting statistical power to identify subgroups more likely to benefit from the intervention. Future trials can further identify populations most likely to benefit from the intervention. Third, surveys were only available in English, Spanish, and Mandarin. Fourth, there was no attention control, and it is possible that the additional attention received by the intervention group, rather than the intervention components, explained the benefits reported here. Fifth, MCIDs are not established for the differences in response rates between groups. Sixth, the trial was not blinded and the outcomes reported herein were patient reported. It is possible that awareness of group assignment contributed to differences between the 2 groups.

Conclusions

In this report of secondary outcomes from a randomized clinical trial of adults with metastatic cancer, use of weekly electronic PRO surveys to monitor symptoms, compared with usual care, resulted in statistically significant improvements in physical function, symptom control, and HRQOL at 3 months, with mean improvements of approximately 2.5 points on a 0- to 100-point scale. These findings should be interpreted provisionally pending results of the primary outcome of overall survival.

Back to top
Article Information

Corresponding Author: Ethan Basch, MD, MSc, Lineberger Comprehensive Cancer Center, University of North Carolina, 170 Manning Dr, Haupt Physician Office Bldg, Chapel Hill, NC 27599 (ebasch@med.unc.edu).

Accepted for Publication: May 17, 2022.

Published Online: June 5, 2022. doi:10.1001/jama.2022.9265

Author Contributions: Dr Dueck had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Basch, Henson, Jansen, Stover, Carr, Jonsson, Rogak, Reeve, Bruner, Perlmutter, Geoghegan, Given, Dueck.

Acquisition, analysis, or interpretation of data: Basch, Schrag, Henson, Jansen, Ginos, Stover, Spears, Jonsson, Deal, Bennett, Thanarajasingam, Rogak, Snyder, Cella, Kottschade, Samuel-Ryals, Mazza, Miller, Strasser, Zylla, Weiss, Blinder, Dueck.

Drafting of the manuscript: Basch, Henson, Stover, Dueck.

Critical revision of the manuscript for important intellectual content: Basch, Schrag, Jansen, Ginos, Stover, Carr, Spears, Jonsson, Deal, Bennett, Thanarajasingam, Rogak, Reeve, Snyder, Bruner, Cella, Kottschade, Perlmutter, Geoghegan, Samuel-Ryals, Given, Mazza, Miller, Strasser, Zylla, Weiss, Blinder, Dueck.

Statistical analysis: Basch, Jansen, Ginos, Deal, Mazza, Dueck.

Obtained funding: Basch, Dueck.

Administrative, technical, or material support: Basch, Schrag, Henson, Jansen, Carr, Jonsson, Rogak, Given, Weiss.

Supervision: Basch, Carr, Geoghegan, Weiss.

Conflict of Interest Disclosures: Dr Basch reported receipt of personal fees from Navigating Cancer, Carevive Systems Inc, Sivan, AstraZeneca, Resilience, and the Research Triangle Institute for serving as a scientific advisor; receipt of research funding from the National Cancer Institute (NCI); being owner of Carolina Informatics; and employment by the University of North Carolina. Dr Schrag reported receipt of personal fees from Pfizer; nonfinancial support from Grail for serving as a site principal investigator; and grants from the American Association for Cancer Research awarded to Memorial Sloan Kettering Cancer Center. Dr Stover reported receipt of personal fees from Navigating Cancer and grants from Sivan and the Bladder Cancer Advocacy Network. Ms Spears reported receipt of personal fees from Pfizer. Dr Snyder reported receipt of grants from Genentech and Pfizer to Johns Hopkins and personal fees from Janssen via Health Outcomes Solutions. Dr Bruner reported receipt of personal fees from Wilmont Cancer Center, the NCI, and FlatIron and grants from the NCI Community Oncology Research Program and the Simons Foundation for Science Gallery Atlanta. Dr Samuel-Ryals reported receipt of grants from the NCI and having an uncompensated advisory role with Voluntis. Dr Weiss reported receipt of payments from Myriad Laboratories Incorporated for a research agreement. Dr Blinder reported receipt of compensation from Carevive Systems Inc. No other disclosures were reported.

Funding/Support: This work was funded by the Patient-Centered Outcomes Research Institute (PCORI) award IHS-1511-33392. The trial made use of technology systems provided by the Patient-Reported Outcomes Core (PRO Core; https://pro.unc.edu) at the Lineberger Comprehensive Cancer Center of the University of North Carolina, funded by NCI Cancer Center Core Support grant 5-P30-CA016086 and the University Cancer Research Fund of North Carolina. The trial was sponsored by Alliance Foundation Trials LLC (https://acknowledgments.alliancefound.org).

Role of the Funder/Sponsor: The sponsor, Alliance Foundation Trials LLC, coordinated contracting with participating study practice sites and submission of regulatory documentation. The sponsor was not involved in the design and conduct of the study; collection, management, analysis, or interpretation of the data; preparation of the manuscript; or decision to submit the manuscript for publication. The Alliance Publications Committee reviewed and approved the manuscript for publication and did not have the right to veto publication or to control the decision regarding to which journal the manuscript was submitted.

Disclaimer: All statements in this publication, including its findings, are solely those of the authors and do not necessarily represent the views of PCORI or its Board of Governors or Methodology Committee. Drs Basch and Schrag are Associate Editors of JAMA but were not involved in any of the decisions regarding review of the manuscript or its acceptance.

Meeting Presentation: Presented at the American Society of Clinical Oncology Annual Meeting; June 5, 2022; Chicago, Illinois.

Data Sharing Statement: See Supplement 3.

Additional Contributions: We express our sincere appreciation to Neeraj Arora, PhD, PCORI, for his invaluable input and guidance throughout the development and conduct of this trial, and to Anna Rapperport, BA, Alliance Foundation Trials LLC, for her indispensable support during the conduct of this trial. Dr Arora was not compensated for his role in this trial specifically. He is compensated by his employer, PCORI, which funded this trial. Ms Rapperport received salary funding from the grant supporting this trial.

References
1.
Cleeland  CS, Zhao  F, Chang  VT,  et al.  The symptom burden of cancer: evidence for a core set of cancer-related and treatment-related symptoms from the Eastern Cooperative Oncology Group Symptom Outcomes and Practice Patterns study.   Cancer. 2013;119(24):4333-4340. doi:10.1002/cncr.28376PubMedGoogle ScholarCrossref
2.
Reilly  CM, Bruner  DW, Mitchell  SA,  et al.  A literature synthesis of symptom prevalence and severity in persons receiving active cancer treatment.   Support Care Cancer. 2013;21(6):1525-1550. doi:10.1007/s00520-012-1688-0PubMedGoogle ScholarCrossref
3.
Panattoni  L, Fedorenko  C, Greenwood-Hickman  MA,  et al.  Characterizing potentially preventable cancer- and chronic disease-related emergency department use in the year after treatment initiation: a regional study.   J Oncol Pract. 2018;14(3):e176-e185. doi:10.1200/JOP.2017.028191PubMedGoogle ScholarCrossref
4.
Henry  DH, Viswanathan  HN, Elkin  EP, Traina  S, Wade  S, Cella  D.  Symptoms and treatment burden associated with cancer treatment: results from a cross-sectional national survey in the US.   Support Care Cancer. 2008;16(7):791-801. doi:10.1007/s00520-007-0380-2PubMedGoogle ScholarCrossref
5.
Di Maio  M, Gallo  C, Leighl  NB,  et al.  Symptomatic toxicities experienced during anticancer treatment: agreement between patient and physician reporting in three randomized trials.   J Clin Oncol. 2015;33(8):910-915. doi:10.1200/JCO.2014.57.9334PubMedGoogle ScholarCrossref
6.
Laugsand  EA, Sprangers  MAG, Bjordal  K, Skorpen  F, Kaasa  S, Klepstad  P.  Health care providers underestimate symptom intensities of cancer patients: a multicenter European study.   Health Qual Life Outcomes. 2010;8:104. doi:10.1186/1477-7525-8-104PubMedGoogle ScholarCrossref
7.
Basch  E, Iasonos  A, McDonough  T,  et al.  Patient versus clinician symptom reporting using the National Cancer Institute Common Terminology Criteria for Adverse Events: results of a questionnaire-based study.   Lancet Oncol. 2006;7(11):903-909. doi:10.1016/S1470-2045(06)70910-XPubMedGoogle ScholarCrossref
8.
Warrington  L, Absolom  K, Conner  M,  et al.  Electronic systems for patients to report and manage side effects of cancer treatment: systematic review.   J Med Internet Res. 2019;21(1):e10875. doi:10.2196/10875PubMedGoogle ScholarCrossref
9.
Kotronoulas  G, Kearney  N, Maguire  R,  et al.  What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? a systematic review of controlled trials.   J Clin Oncol. 2014;32(14):1480-1501. doi:10.1200/JCO.2013.53.5948PubMedGoogle ScholarCrossref
10.
Basch  E, Deal  AM, Kris  MG,  et al.  Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial.   J Clin Oncol. 2016;34(6):557-565. doi:10.1200/JCO.2015.63.0830PubMedGoogle ScholarCrossref
11.
Basch  E, Deal  AM, Dueck  AC,  et al.  Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment.   JAMA. 2017;318(2):197-198. doi:10.1001/jama.2017.7156PubMedGoogle ScholarCrossref
12.
Denis  F, Basch  E, Septans  AL,  et al.  Two-year survival comparing web-based symptom monitoring vs routine surveillance following treatment for lung cancer.   JAMA. 2019;321(3):306-307. doi:10.1001/jama.2018.18085PubMedGoogle ScholarCrossref
13.
Barbera  L, Sutradhar  R, Seow  H,  et al.  Impact of standardized Edmonton Symptom Assessment System use on emergency department visits and hospitalization: results of a population-based retrospective matched cohort analysis.   JCO Oncol Pract. 2020;16(9):e958-e965. doi:10.1200/JOP.19.00660PubMedGoogle ScholarCrossref
14.
Mir  O, Ferrua  M, Fourcade  A,  et al.  Digital remote monitoring plus usual care versus usual care in patients treated with oral anticancer agents: the randomized phase 3 CAPRI trial.   Nat Med. Published online April 25, 2022. doi:10.1038/s41591-022-01788-1PubMedGoogle ScholarCrossref
15.
Schmidt  T, Valuck  T, Perkins  B,  et al.  Improving patient-reported measures in oncology: a payer call to action.   J Manag Care Spec Pharm. 2021;27(1):118-126. doi:10.18553/jmcp.2020.20313PubMedGoogle ScholarCrossref
16.
Samuel  CA, Schaal  J, Robertson  L,  et al.  Racial differences in symptom management experiences during breast cancer treatment.   Support Care Cancer. 2018;26(5):1425-1435. doi:10.1007/s00520-017-3965-4PubMedGoogle ScholarCrossref
17.
Patient-Reported Outcomes Core of the University of North Carolina. Accessed May 27, 2022. https://pro.unc.edu/
18.
Basch  E, Stover  AM, Schrag  D,  et al.  Clinical utility and user perceptions of a digital system for electronic patient-reported symptom monitoring during routine cancer care: findings from the PRO-TECT trial.   JCO Clin Cancer Inform. 2020;4:947-957. doi:10.1200/CCI.20.00081PubMedGoogle ScholarCrossref
19.
Schoen  MW, Basch  E, Hudson  LL,  et al.  Software for administering the National Cancer Institute’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events: usability study.   JMIR Hum Factors. 2018;5(3):e10070. doi:10.2196/10070PubMedGoogle ScholarCrossref
20.
Basch  E, Barbera  L, Kerrigan  CL, Velikova  G.  Implementation of patient-reported outcomes in routine medical care.   Am Soc Clin Oncol Educ Book. 2018;38:122-134. doi:10.1200/EDBK_200383PubMedGoogle ScholarCrossref
21.
National Cancer Institute. Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events. Accessed May 27, 2022. https://healthcaredelivery.cancer.gov/pro-ctcae
22.
Dueck  AC, Mendoza  TR, Mitchell  SA,  et al; National Cancer Institute PRO-CTCAE Study Group.  Validity and reliability of the US National Cancer Institute’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE).   JAMA Oncol. 2015;1(8):1051-1059. doi:10.1001/jamaoncol.2015.2639PubMedGoogle ScholarCrossref
23.
Al-Rashdan  A, Sutradhar  R, Nazeri-Rad  N, Yao  C, Barbera  L.  Comparing the ability of physician-reported versus patient-reported performance status to predict survival in a population-based cohort of newly diagnosed cancer patients.  2021;33(7):476-482.  Clin Oncol (R Coll Radiol). doi:10.1016/j.clon.2021.01.008PubMedGoogle ScholarCrossref
24.
de Souza  JA, Yap  BJ, Wroblewski  K,  et al.  Measuring financial toxicity as a clinically relevant patient-reported outcome: the validation of the Comprehensive Score for Financial Toxicity (COST).   Cancer. 2017;123(3):476-484. doi:10.1002/cncr.30369PubMedGoogle ScholarCrossref
25.
Aaronson  NK, Ahmedzai  S, Bergman  B,  et al.  The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology.   J Natl Cancer Inst. 1993;85(5):365-376. doi:10.1093/jnci/85.5.365PubMedGoogle ScholarCrossref
26.
Fayers  P, Bottomley  A; EORTC Quality of Life Group; Quality of Life Unit; European Organisation for Research and Treatment of Cancer.  Quality of life research within the EORTC—the EORTC QLQ-C30.   Eur J Cancer. 2002;38(suppl 4):S125-S133. doi:10.1016/S0959-8049(01)00448-8PubMedGoogle ScholarCrossref
27.
Gundy  CM, Fayers  PM, Groenvold  M,  et al.  Comparing higher order models for the EORTC QLQ-C30.   Qual Life Res. 2012;21(9):1607-1617. doi:10.1007/s11136-011-0082-6PubMedGoogle ScholarCrossref
28.
Fayers  PM, Aaronson  NK, Bjordal  K, Groenvold  M, Curran  D, Bottomley  A; EORTC Quality of Life Group.  The EORTC QLQ-C30 Scoring Manual (3rd Edition). European Organisation for Research and Treatment of Cancer; 2001.
29.
Cocks  K, King  MT, Velikova  G,  et al.  Evidence-based guidelines for interpreting change scores for the European Organisation for the Research and Treatment of Cancer Quality of Life Questionnaire Core 30.   Eur J Cancer. 2012;48(11):1713-1721. doi:10.1016/j.ejca.2012.02.059PubMedGoogle ScholarCrossref
30.
Coens  C, Pe  M, Dueck  AC,  et al; Setting International Standards in Analyzing Patient-Reported Outcomes and Quality of Life Endpoints Data Consortium.  International standards for the analysis of quality-of-life and patient-reported outcome endpoints in cancer randomised controlled trials: recommendations of the SISAQOL Consortium.   Lancet Oncol. 2020;21(2):e83-e96. doi:10.1016/S1470-2045(19)30790-9PubMedGoogle ScholarCrossref
31.
Osoba  D, Rodrigues  G, Myles  J, Zee  B, Pater  J.  Interpreting the significance of changes in health-related quality-of-life scores.   J Clin Oncol. 1998;16(1):139-144. doi:10.1200/JCO.1998.16.1.139PubMedGoogle ScholarCrossref
32.
Basch  E, Abernethy  AP.  Encouraging clinicians to incorporate longitudinal patient-reported symptoms in routine clinical practice.   J Oncol Pract. 2011;7(1):23-25. doi:10.1200/JOP.2010.000186PubMedGoogle ScholarCrossref
33.
Lizée  T, Basch  E, Trémolières  P,  et al.  Cost-effectiveness of web-based patient-reported outcome surveillance in patients with lung cancer.   J Thorac Oncol. 2019;14(6):1012-1020. doi:10.1016/j.jtho.2019.02.005PubMedGoogle ScholarCrossref
34.
Carlotto  A, Hogsett  VL, Maiorini  EM, Razulis  JG, Sonis  ST.  The economic burden of toxicities associated with cancer treatment: review of the literature and analysis of nausea and vomiting, diarrhoea, oral mucositis and fatigue.   Pharmacoeconomics. 2013;31(9):753-766. doi:10.1007/s40273-013-0081-2PubMedGoogle ScholarCrossref
35.
Centers for Medicare & Medicaid Services. Informal request for information on proposed Oncology Care First model. Accessed May 27, 2022. https://innovation.cms.gov/Files/x/ocf-informalrfi.pdf
36.
National Academies of Sciences, Engineering, and Medicine. Establishing Effective Patient Navigation Programs in Oncology: Proceedings of a Workshop. National Academies Press; 2018.
37.
Stover  AM, Henson  S, Jansen  J,  et al.  Demographic and symptom differences in PRO-TECT trial (AFT-39) cancer patients electing to complete weekly home patient-reported outcome measures (PROMs) via an automated phone call vs. email: implications for implementing PROs into routine care.   Qual Life Res. 2019;28(suppl 1):S1.Google Scholar
×