Differences noted in reports on pain (A), dysphagia (B), nausea (C), and vomiting (D) are plotted against the mean of the PRO-CTCAE score (score range, 0 [none] to 4 [very severe]) and the CTCAE grade (score range, 1 [mild] to 5 [death]) from 327 survey pairs. The large bubble point (position 0, 0) for all symptoms indicates a high level of agreement between patients and practitioners when toxic effects are absent. As the toxic effects increase (x-axis), the number of agreements between patients and practitioners diminishes, as demonstrated by the decreasing size of the bubbles and their greater distance from the x-axis. Bubble points lying directly on the x-axis indicate that patient-reported severity tended to be greater than the corresponding toxic effect grade provided by the practitioner.
eFigure. Agreement and Disagreement Between Patient-Reported and Practitioner-Reported Symptom Severity
Customize your JAMA Network experience by selecting one or more topics from the list below.
Falchook AD, Green R, Knowles ME, et al. Comparison of Patient- and Practitioner-Reported Toxic Effects Associated With Chemoradiotherapy for Head and Neck Cancer. JAMA Otolaryngol Head Neck Surg. 2016;142(6):517–523. doi:10.1001/jamaoto.2016.0656
Agreement between patient- and practitioner-reported toxic effects during chemoradiotherapy for head and neck cancer is unknown.
To compare patient-reported symptom severity and practitioner-reported toxic effects among patients receiving chemoradiotherapy for head and neck cancer.
Design, Setting, and Participants
Forty-four patients participating in a phase 2 trial of deintensified chemoradiotherapy for oropharyngeal carcinoma were included in the present study (conducted from February 8, 2012, to March 2, 2015). Most treatment (radiotherapy, 60 Gy, with concurrent weekly administration of cisplatin, 30 mg/m2) was administered at academic medical centers. Included patients had no prior head and neck cancers, were 18 years or older, and had a smoking history of 10 pack-years or less or more than 10 pack-years but 30 pack-years or less and abstinent for the past 5 years. Cancer status was untreated human papillomavirus or p16-positive squamous cell carcinoma of the oropharynx or unknown head and neck primary site; and cancer staging was category T0 to T3, category N0 to N2c, M0, and Eastern Cooperative Oncology Group performance status 0 to 1. Baseline, weekly, and posttreatment toxic effects were assessed by physicians or nurse practitioners using National Cancer Institute’s Common Terminology Criteria for Adverse Events (CTCAE), version 4.0. Patient-reported symptom severity was measured using the Patient-Reported Outcomes version of the CTCAE (PRO-CTCAE). Descriptive statistics were used to characterize raw agreement between CTCAE grades and PRO-CTCAE severity ratings.
Baseline, weekly, and posttreatment toxic effects assessed using CTCAE, version 4.0, and PRO-CTCAE.
Main Outcomes and Measures
Raw agreement indices between patient-reported toxic effects, including symptom frequency, severity, and interference with daily activities (score range, 0 [none] to 4 [very severe]), and practitioner-measured toxic effects, including swallowing, oral pain, and hoarseness (score range, 1 [mild] to 5 [death]).
Of the 44 patients included in the analysis (39 men, 5 women; mean [SD] age, 61 [8.4] years), there were 327 analyzable pairs of CTCAE and PRO-CTCAE symptom surveys and no treatment delays due to toxic effects. Patient-reported and practitioner-reported symptom severity agreement was high at baseline when most symptoms were absent but declined throughout treatment as toxic effects increased. Most disagreement was due to lower severity of toxic effects reported by practitioners (eg, from 45% agreement at baseline to 27% at the final week of treatment for pain). This was particularly noted for domains that are not easily evaluated by physical examination, such as anxiety and fatigue (eg, severity of fatigue decreased from 43% at baseline to 12% in the final week of treatment).
Conclusions and Relevance
Practitioner-reported toxic effects are lower than patient self-reports during head and neck chemoradiotherapy. The inclusion of patient-reported symptomatic toxic effects provides information that can potentially enhance clinical management and improve data quality in clinical trials.
Accurate assessment of toxic effects is an essential function in cancer clinical trials and routine care of cancer patients. In clinical trials, investigators commonly evaluate and quantify toxic effects and adverse events using the Common Terminology Criteria for Adverse Events (CTCAE) instrument developed by the National Cancer Institute.1 Emerging evidence suggests that direct patient reporting of symptoms may improve the accuracy and validity of toxic effects assessment.2 Disagreement between patient-reported and practitioner-reported symptomatic toxic effects has been extensively described3 and between-practitioner agreement in toxic effects ratings may also be only modest.4 There is evidence5,6 to suggest that, compared with practitioner reporting, patient-reported symptoms are more strongly correlated with clinical outcomes.
To improve the reliability of capturing treatment-related adverse events, the National Cancer Institute has developed a library of patient-reported outcome (PRO) measures that complement the CTCAE. The PRO version of the CTCAE (PRO-CTCAE) allows patients to self-report the frequency, severity, and activity interference of symptoms that they are experiencing while undergoing treatment.7 Implementation of the PRO-CTCAE may improve the quality of adverse event data in clinical trials and foster communication between patients and practitioners.8 The PRO-CTCAE has demonstrated2 favorable validity, reliability, and responsiveness in patients undergoing cancer treatment.
The use of patient-reported measures has been recommended9 in head and neck cancer treatment trials. Head and neck cancers develop in tissues with close proximity to vital anatomical structures that are involved in important physiologic (swallowing and eating) and social (communication) functions; thus, the prevalence, severity, and impact of treatment-related toxic effects is high.10 Chemoradiotherapy is a standard and intensive treatment for cancers of the head and neck, with approximately 50% of the patients experiencing severe to very severe acute and late toxic effects.10-12 Although chemoradiotherapy for head and neck cancer is an outpatient treatment, 20% of patients will have an unplanned hospital admission.13 The National Cancer Institute Symptom Management and Quality of Life Steering Committee conducted a clinical trials planning meeting to identify a standard core set of PRO symptoms to be assessed across all disease sites and in head and neck cancer clinical trials.14 This core set of symptoms is also included in the PRO-CTCAE item library.
We have incorporated a subset of PRO-CTCAE items to assess the National Cancer Institute core symptoms into our multi-institutional head and neck chemoradiotherapy trials. To our knowledge, there are no published data regarding the use of PRO-CTCAE (specifically, how PRO-CTCAE reports compare with CTCAE assessments) in head and neck clinical trials. The purpose of this study was to compare patient reporting (PRO-CTCAE) with practitioner reporting (CTCAE) of symptom severity using data derived from a multi-institutional clinical trial15 of chemoradiotherapy in patients with head and neck cancer.
Question How do patient-reported symptoms compare with practitioner-reported toxic effects among patients receiving chemoradiotherapy for head and neck cancer?
Findings Agreement between patients and practitioners was high at baseline but decreased throughout the course of treatment. Most of the disagreement involved lower severity of the toxic effects reported by practitioners.
Meaning The inclusion of patient-reported data helps to provide a more accurate and complete picture of toxic effects that patients experience during cancer treatment.
We conducted an analysis of patients enrolled in a prospective phase 2 clinical trial (NCT01530997).15 The primary objective of the phase 2 study was to evaluate the efficacy of a deintensified chemoradiotherapy regimen in favorable-risk, human papilloma virus (HPV)–associated oropharyngeal squamous cell carcinoma. The study, including the present analysis of patient- and practitioner-reported toxic effects, received institutional review board approval at the participating centers. All patients provided written informed consent; no financial compensation was provided. Enrolling institutions included the University of North Carolina Hospitals, Chapel Hill; University of Florida Hospitals, Gainesville; and Rex Hospital, Raleigh, North Carolina. The study was conducted from February 8, 2012, to March 2, 2015.
Eligible patients (age, ≥18 years; all native English speakers) had a smoking history of 10 pack-years or less or more than 10 pack-years but 30 pack-years or less and had been abstinent for the past 5 years; adequate hematologic, renal, and liver function; no prior head and neck cancers; and untreated, pathologically confirmed HPV- or p16-positive squamous cell carcinoma of the oropharynx or from an unknown head and neck primary site that was category T0 to 3, category N0 to N2c, M0, and Eastern Cooperative Oncology Group performance status 0 to 1.
All patients received intensity-modulated radiotherapy delivered to a total dose of 60 Gy at 2 Gy per fraction 5 days per week for 6 weeks. A dose of 54 Gy was delivered to anatomical regions at risk for subclinical disease. Concurrent weekly low-dose cisplatin (30 mg/m2) was administered, with dose modifications allowed as needed. Within 6 to 14 weeks after chemoradiotherapy, all patients underwent planned surgical evaluation, which consisted of a biopsy of the primary site and dissection of pretreatment-positive lymph node regions.15
The core set of symptoms recommended for assessment in head and neck clinical trials includes difficulty swallowing, oral pain, skin changes, dry mouth, lowered dental health, difficulty with opening mouth or trismus, impaired taste, excess or thick mucus or saliva, shoulder disability or impaired motion, and altered voice or hoarseness.10 Additional symptoms recommended for assessment across all disease sites are fatigue, insomnia, pain, anorexia (appetite loss), dyspnea, cognitive problems, anxiety (includes worry), nausea, depression (sad and unhappy feelings), sensory neuropathy, constipation, and diarrhea.14 Dental health and shoulder disability or impaired motion are health states rather than symptomatic toxic effects; thus, they are not evaluated using PRO-CTCAE. Furthermore, symptoms such as insomnia, dyspnea, cognitive problems, and sensory neuropathy occur infrequently with this regimen.
Criteria for grading on the CTCAE scale vary by toxic effects; however, grade 1 typically refers to mild symptoms not requiring intervention, grade 2 refers to moderate symptoms that interfere somewhat with daily function and where some intervention may be indicated, and grade 3 refers to severe symptoms that interfere with daily activities or require more significant intervention. Grade 4 toxic effects (life-threatening, with urgent intervention indicated) and grade 5 (death) are indicative of significantly higher levels of toxic effects and are not applicable for symptomatic effects such as fatigue, tinnitus, hoarseness, or xerostomia. A patient-reported symptom rated as severe or very severe may not correspond to CTCAE grade 4 or 5 toxic effects; however, severe levels of patient-reported toxic effects may indicate a higher risk of impending serious adverse events.6
The PRO-CTCAE measurement system includes questions related to symptom frequency, severity, and interference with daily activities (none, mild, moderate, and severe/very severe), as well as presence/absence and amount, as relevant to the symptom under consideration.7
PRO-CTCAE items reflecting the 13 symptomatic toxic effects chosen for surveillance in this study were assessed via an online questionnaire (Qualtrics LLC). The study coordinator (including R.G.) provided participants with a tablet computer during scheduled weekly visits. PRO-CTCAE and CTCAE assessments were obtained before treatment, weekly during chemoradiotherapy, and at subsequent follow-up visits.
Practitioner-reported toxicity was graded using CTCAE from 1 (mild) to 5 (death). For domains in which the practitioner determined that a toxicity was not present, a grade of 0 was assigned. No patients experienced grade 4 (life-threatening) or grade 5 (death) toxic effects; these grades were removed from the CTCAE assessment scale for purposes of comparison with PRO-CTCAE scores. Therefore, the CTCAE scores had a restricted range from 0 (none) to 3 (severe). Severity of symptoms using the PRO-CTCAE was scored as follows: 0 (none), 1 (mild), 2 (moderate), 3 (severe), and 4 (very severe). To facilitate direct comparison between CTCAE and PRO-CTCAE, patient-reported symptoms rated as severe and very severe were classified together as severe.
The analysis reported here examines comparisons between CTCAE grades and PRO-CTCAE symptom severity scores. For the 5 PRO-CTCAE symptom terms where a frequency question is asked first (pain, anxiety, depressed mood, nausea, and vomiting), if a patient indicated no symptom frequency, then a score of none was assigned for severity for the purposes of this analysis. The timing of the practitioner assessments of toxic effects was uniform across all institutions in this study. Assessments were made by physicians or nurse practitioners after weekly clinic visits without knowledge of the PRO-CTCAE reports for that assessment time point or prior time points.
Descriptive statistics were used to summarize the patient cohort demographics and disease characteristics. Results of toxic effects assessments using CTCAE and PRO-CTCAE were summarized at baseline and for each week of treatment. For each symptom assessment time point and each symptom domain, we measured the raw agreement between CTCAE grade and PRO-CTCAE symptom severity. We then analyzed the proportion of pairs with symptomatic adverse events scored or graded as more severe by patients or practitioners, as well as the degree of discrepancy. Agreement between practitioners and patients was defined as the same severity of toxic effects reported by practitioners using the CTCAE scale (grade 0, 1, 2, or 3) and patients using the PRO-CTCAE (none, mild, moderate, and severe/very severe), for a given time point. To further evaluate how increasing symptom severity may impact differences between CTCAE grades and PRO-CTCAE severity scores, Bland-Altman plots were constructed for the following symptom domains: pain, dysphagia, nausea, and vomiting. These domains were chosen because under-ascertainment of toxic effects by practitioners for these symptoms could lead to adverse events, such as emergency department visits or unplanned hospitalization. All statistical analyses were performed using SAS, version 9.4 (SAS Institute Inc).
There were evaluable data for 44 of 45 patients (98%) enrolled in this phase 2 clinical trial (Table 1). Thirty-nine patients (89%) were male and 40 patients (91%) were white. All patients completed the full course of radiotherapy (60 Gy), and 31 individuals (70%) received the planned 6 weekly doses of cisplatin. There were no radiotherapy treatment delays related to toxic effects. Surgical evaluation was performed for 43 patients (1 patient lost to follow-up did not undergo surgery) at a mean (SD) of 9 (2) weeks (range, 7-14 weeks) after completion of radiotherapy. Seventeen patients (39%) required placement of a percutaneous gastrostomy tube for a mean duration of 15 (4) weeks (range, 5-22 weeks). No patient required a long-term gastrostomy tube. All patients were alive with no evidence of disease (median follow-up, 21 months, range, 4-41 months). Thirty-eight patients (86%) had a follow-up of at least 1 year.
There were 352 scheduled visits at which patients were expected to self-report symptoms using PRO-CTCAE. Complete PRO-CTCAE data were available for 335 of those visits, yielding an adherence rate of 95.2%. At these same visits, practitioners submitted 342 CTCAE reports (adherence rate, 97.2%). There were 25 instances (of 352 of planned assessments) where either CTCAE or PRO-CTCAE data were unavailable for comparison, resulting in 327 (92.9%) analyzable pairs of CTCAE and PRO-CTCAE symptom reports at baseline, during treatment, and 6 months after treatment for the 44 patients. Raw indices of agreement between practitioner-reported and patient-reported symptom severity are reported in Table 2. The highest agreement was noted at baseline (before treatment), with agreement declining throughout the course of treatment (eFigure in the Supplement). An overall decrease in practitioner and patient agreement was seen for every symptom domain across the course of treatment. For example, agreement in the severity of fatigue was 43% at baseline and 12% in the final week of treatment.
Toxic effects due to chemoradiotherapy are cumulative and typically are greatest at the end of treatment. A summary of the PRO-CTCAE scores and CTCAE grades for symptom toxic effects in the final week of treatment is given in Table 3. Practitioner-reported CTCAE grades were generally lower than PRO-CTCAE scores, especially for symptom domains not easily evaluable by physical examination. For example, fatigue was reported as mild by 6 patients (14%), but practitioners reported fatigue as none or mild for 36 patients (82%). There was greater agreement for symptomatic toxic effects, such as radiation-associated dermatitis and mucositis, although a similar pattern of discrepancy between practitioner grades and patient severity scores was present in these domains as well.
Bland-Altman plots for pairs of CTCAE and PRO-CTCAE symptom reports for pain, dysphagia, nausea, and vomiting are shown in the Figure. These plots demonstrate that greater disagreement between practitioners and patients was observed, with practitioners reporting lower CTCAE grades compared with patient-reported PRO-CTCAE severity scores.
The present study reports a direct comparison between CTCAE grades and PRO-CTCAE scores for patients with head and neck cancer who are receiving chemoradiotherapy. Although there is apparent similarity between grading scales for PRO-CTCAE and CTCAE, differences between CTCAE grades and PRO-CTCAE scores should not be overinterpreted. Disagreement between PRO-CTCAE and CTCAE toxic effects assessment may not represent under-ascertainment by practitioners or overreporting by patients. For example, a patient may meet criteria for grade 1 vomiting by CTCAE criteria (1 or 2 episodes separated by 5 minutes in a 24-hour period), but to the patient, 2 daily episodes of vomiting constitutes a severe symptom. Frequent management of severe toxic effects by practitioners may skew practitioners’ perceptions of symptom severity. The severity of chemoradiotherapy-associated toxic effects represents a potential source of discordance between practitioner-reported and patient-reported symptoms (eg, daily vomiting alone not categorized as a high-grade toxic effect by the practitioner, but considered quite severe by the patient). There is no objective standard for a correct ascertainment of symptom severity. The lack of an objective standard supports the goal of collecting as much information as possible regarding symptoms from both patients and practitioners to obtain the most complete and accurate picture of toxic effects.
A substantial percentage of the disagreement between practitioner- and patient-reported symptoms reflected disagreement by 1 point. The clinical impact of these small differences likely depends on the domain being assessed as well as the extent of symptom severity. A disagreement between mild fatigue and no fatigue is not likely to affect clinical management or patient outcomes, but a difference between moderate dysphagia and severe dysphagia could have potentially significant implications regarding patient nutrition or the possible need for aggressive intervention, such as gastric tube placement. In the present study, it was not possible to directly evaluate the association between discordant ratings and patient outcomes because there were few severe adverse events (eg, unplanned hospitalization, placement of feeding tube, and death) in this sample of patients receiving a deintensified regimen of chemoradiotherapy for oropharyngeal cancer.
There are unique challenges when attempting to directly compare CTCAE and PRO-CTCAE data. These scoring instruments were not developed for direct comparison, but rather were designed to be complementary strategies for capturing symptomatic adverse events. A limitation of the present study is the relatively low number of practitioner assessors of toxic effects. However, CTCAE grades are defined by explicit criteria, which should promote uniformity in toxic effects grading by different assessors.
Both PRO-CTCAE and CTCAE measurement systems evaluate symptomatic toxic effects relative to symptom severity, frequency, and interference with activities. In addition, CTCAE grading criteria incorporate the need for clinical intervention. Depending on the symptom under consideration, all 3 factors may be assessed in the PRO-CTCAE and CTCAE scales, which can complicate efforts to make comparisons. For example, in the evaluation of vomiting, frequency and severity are assessed by PRO-CTCAE, whereas frequency and the need for clinical intervention are reflected in CTCAE grading criteria. As a consequence, the present analysis compared PRO-CTCAE vomiting severity with CTCAE vomiting grade.
The direct comparison between CTCAE and PRO-CTCAE in this study serves to highlight patient perceptions regarding their symptoms due to head and neck cancer. For example, more than 30% of the patients reported some degree of xerostomia and/or mucositis at baseline. These symptoms are not readily attributable to untreated oropharyngeal cancer. Similarly, a high degree of hoarseness was reported at baseline and increased throughout treatment. Patients in this trial did not have laryngeal involvement with cancer, and the radiation dose to the larynx was minimized during radiotherapy planning. This finding suggests that it may be difficult for patients to precisely localize their symptoms when communicating with practitioners.
In the present study, practitioner assessments of toxic effects were made without knowledge of patient-reported symptoms. As PRO-CTCAE continues to be incorporated into clinical practice, there is an obvious benefit in systematically providing practitioners with patient-reported data, which may contribute to their assessment of toxic effects. Furthermore, weekly or other periodic assessment of PRO-CTCAE may be supplemented by a real-time symptom monitoring approach. The use of Internet and mobile devices that allow patients to report symptoms as they occur could provide additional insight into the timing and severity of symptoms.16-18
Chemoradiotherapy-associated toxic effects that developed in treatment of head and neck cancer reported in this study may be lower owing to the dose reduction used in the deintensified regimen. In addition, studies19,20 suggest that patients with non–HPV-associated head and neck cancer may experience worse symptoms compared with patients with HPV-associated tumors. It is unclear what effect tumor HPV status may have on agreement between patient- and practitioner-reported toxic effects. The patients in this study were participants in a phase 2 clinical trial at institutions with dedicated head and neck cancer teams, and the level of supportive care provided to patients was similar at all institutions. Studies21-24 have shown improved oncologic outcomes for patients with head and neck cancer treated with multimodality therapy at high-volume centers compared with lower-volume centers. Findings from this study may not apply to patients with oropharyngeal cancer treated in community centers.
The severity of symptoms ascertained by practitioners was lower than that reported by patients, and these differences increased throughout a course of chemoradiotherapy for oropharyngeal cancer. The clinical implications of these differences are unknown. The inclusion of patient-reported data helps to provide a more complete picture of toxic effects that patients experience during cancer treatment.
Corresponding Author: Bhishamjit S. Chera, MD, Department of Radiation Oncology, University of North Carolina, 101 Manning Dr, CB 7512, Chapel Hill, NC 27599 (email@example.com).
Accepted for Publication: March 5, 2016.
Published Online: May 5, 2016. doi:10.1001/jamaoto.2016.0656.
Author Contributions: Dr Chera had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Falchook, Knowles, Basch, Chera.
Acquisition, analysis, or interpretation of data: Falchook, Green, Knowles, Mendenhall, Hayes, Grilley-Olson, Weiss, Reeve, Mitchell, Basch, Chera.
Drafting of the manuscript: Falchook, Green, Mendenhall, Hayes, Basch, Chera.
Critical revision of the manuscript for important intellectual content: Falchook, Knowles, Amdur, Hayes, Grilley-Olson, Weiss, Reeve, Mitchell, Basch, Chera.
Statistical analysis: Falchook, Green, Chera.
Obtained funding: Chera.
Administrative, technical, or material support: Knowles, Hayes, Weiss, Mitchell, Basch, Chera.
Study supervision: Knowles, Amdur, Mendenhall, Hayes, Weiss, Basch, Chera.
Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Weiss reported receiving personal fees from AstraZeneca and Biodesix and grants from Astellas, AstraZeneca, Celgene, Merck, and Novartis outside the submitted work. No other disclosures were reported.
Previous Presentations: This work was given in part as an oral presentation at the American Society for Radiation Oncology Annual Meeting; October 19, 2015; San Antonio, Texas.
Create a personal account or sign in to: