Each circle represents a participating Institution. The size of each circle represents the number of patients enrolled in the study. Correlations for fatigue, pain, paresthesia, and dyspnea are shown in eFigure 3 in the Supplement.
eTable 1. Participating Centers and corresponding contribution to the study
eTable 2. Patient demographics
eTable 3. Incomplete information in the patient questionnaires (%)
eTable 4. Summary of the results of paired patient questionnaires
eTable 5. Summary of the results of paired doctor questionnaires
eTable 6. Summary of the results of paired patient and doctor questionnaires after cycle 1
eTable 7. Cohen κ and weighted Cohens κ statistics between patient’s and doctor’s questionnaires computed on the 5-level toxicity ordinal scale
eFigure 1. Sample of one item (nausea ) of the Italian questionnaire
eFigure 2. Discrepancy in toxicity grading between patients and doctors (first questionnaire)
eFigure 3. Correlation between discrepancy in patient and doctors reporting of toxicity grade (fatigue, pain, paresthesia and dyspnea) and number of patients enrolled at each center (first questionnaire). Each circle represents a participating Institution. The size of each circle represents the number of patients enrolled in the study.
eAppendix. English translation of the 10-item questionnaire
Customize your JAMA Network experience by selecting one or more topics from the list below.
Montemurro F, Mittica G, Cagnazzo C, et al. Self-evaluation of Adjuvant Chemotherapy-Related Adverse Effects by Patients With Breast Cancer. JAMA Oncol. 2016;2(4):445–452. doi:10.1001/jamaoncol.2015.4720
Patient perspective on chemotherapy-related adverse effects is being increasingly acknowledged both in experimental clinical trials and in clinical practice.
To evaluate a 10-item, paper questionnaire derived from the US National Cancer Institute’s Common Terminology Criteria for Adverse Events (CTCAE) version 4.0 for patient-reported chemotherapy-related adverse effects.
Design, Setting, and Participants
Prospective, single-arm study of 604 women with breast cancer receiving standard adjuvant chemotherapy conducted at 11 outpatient oncology clinics at academic and nonacademic Italian hospitals between January 2011 and October 2013. The CTCAE version 4.0 definitions of grade of severity for nausea, vomiting, constipation, anorexia, dysgeusia, diarrhea, fatigue, pain, paresthesia, and dyspnea were translated into Italian and rephrased. Questionnaires were administered after the first and third cycle of chemotherapy. Adverse effect information was also extracted from the medical records to compare with patient-reported data.
Main Outcomes and Measures
Differences in adverse effect-reporting between paired questionnaires and agreement between patient and physician adverse effect-reporting (grade 0 vs grade ≥1) were studied. Linear regression was used to study the effect of the number of patients enrolled at each institution on the magnitude of discrepancy in adverse effect-reporting between patients and physicians.
A total of 604 women (median age, 53.4 years; interquartile range, 45.0-62.7 years) were enrolled. The number of patients enrolled at each site varied between 6 and 236. Three patients withdrew consent prior to starting the first cycle of adjuvant chemotherapy. After cycle 1 of adjuvant chemotherapy, 596 patient questionnaires were collected, and 581 patient questionnaires were collected after cycle 3. Of the questionnaires collected, 594 and 573 had corresponding questionnaire results extracted from medical records at the same time point. The median (interquartile range) percentage of completed questionnaire fields was 82% (80%-88%) for both the first and third cycle questionnaires, and the results of the 2 patient questionnaires showed a reduction in vomiting (severity), diarrhea (both incidence and severity), and pain (both incidence and severity), as well as a statistically significant increase in dysgeusia (both incidence and severity) and dyspnea (both incidence and severity) in the second patient-completed questionnaire. The frequency and severity of chemotherapy-related adverse effects were consistently greater in patient-reported data than physician-reported data. As a result, interrater agreement was low for most adverse effects, ranging from 0.10 for anorexia to 0.54 for vomiting (Cohen κ statistic). There was a strong and significant positive correlation between the magnitude of the discrepancy in the frequency of reporting adverse effects and the number of patients enrolled at each site.
Conclusions and Relevance
Adherence to reporting adjuvant chemotherapy-related adverse effects using the CTCAE system is high in women undergoing adjuvant chemotherapy for breast cancer. Workload may contribute to agreement discrepancies by limiting the physician-patient relationship.
Accurate reporting of treatment-related adverse effects is crucial for the appropriate management of patients with cancer. In routine clinical practice, adverse-effect reporting is pivotal in the decision-making process for dose modifications (eg, dose reduction, treatment delay, or withdrawal), supportive care administration, and prophylaxis. Adverse-effect reporting (occurrence and grade of severity) is mandatory and highly standardized in clinical trials of new drugs or trials that compare newer treatments or regimens with an established gold standard.1-3 The National Cancer Institute (NCI) Common Terminology Criteria for Adverse Events (CTCAE) is the most widely used protocol for the description and grading of treatment-related adverse effects in cancer therapy trials,4 and CTCAE items include laboratory abnormalities, objective clinical findings, and subjective adverse effects experienced by the patient during and/or after drug exposure. The collection, coding, and grading of patient-reported adverse events is usually based on information abstracted from the patient’s medical records, making the use of a standardized system like the CTCAE demanding and resource-intensive for the clinical research team. Furthermore, the patient experience may not be fully captured by the physicians or nurses documenting clinical findings. Several articles5-11 from clinical trials have highlighted discrepancies in the adverse effects reported by medical staff and patients, with physicians frequently underestimating the incidence and severity of adverse effects. Since underreporting of chemotherapy-related adverse effects is a problem even in the highly controlled clinical trial environment, it is likely to be even more pronounced in routine clinical practice, where reporting does not follow strict standards. Consequently, there is growing interest in including patient-reported outcomes (PROs) in a CTCAE-compliant manner, where PROs are subjective reports from patients without medical interpretation (eg, quality-of-life questionnaires).12-18
Because the use of a standard reporting system like CTCAE in routine clinical practice is attractive yet challenging to implement,19 we established a prospective study in mid-2010 to evaluate how patients with breast cancer receiving standard adjuvant chemotherapy report and grade adverse effects using a 10-item, paper questionnaire. The questionnaire was derived from the CTCAE version 4.0 definitions and translated to Italian. The adverse effects reported by patients were also compared with those described by the treating oncologists who were informed of, but not formally involved in, the study. To minimize bias from cancer-associated symptoms, women with nonmetastatic breast cancer undergoing standard adjuvant chemotherapy after breast surgery were studied.
We evaluated whether patients undergoing conventional chemotherapy could report and grade adverse effects according to the Common Terminology Criteria for Adverse Events (CTCAE) version 4.0.
A 10-item, CTCAE version 4.0–derived Italian questionnaire was administered to 601 breast cancer patients after the first and third cycle of adjuvant chemotherapy. Physician and patient reports were compared.
The return rate was 99% and 95% for the first and second questionnaire, and the median percentage of completed fields was 82% for both questionnaires.
Compared with patients, physicians systematically underestimated incidence for all 10 adverse effects on the questionnaire, resulting in concordance rates ranging from 0.10 to 0.54 (Cohen κ statistic). Similarly, severity (grade) was underestimated by physicians.
Discrepancy in adverse-effect reporting correlated positively with the number of patients enrolled at each site, suggesting patient workload has an effect on this phenomenon that deserves further investigation.
Women 18 years and older who had undergone surgery for operable breast cancer and were candidates for first-time adjuvant chemotherapy outside a clinical trial were eligible for study. Patients were asked to fill in the questionnaire at the end of the first chemotherapy cycle (usually on the day scheduled for the second cycle of chemotherapy) and at the end of the third cycle (usually on the day scheduled for the fourth cycle of chemotherapy). Patients completed the questionnaires in the waiting room. Because it has previously been shown that the timing of questionnaire completion with respect to seeing the oncologist does not introduce significant bias,5 patients were able to complete the questionnaires before or after visiting the treating oncologist but before administration of chemotherapy. Oncology staff provided no assistance with filling in the questionnaires, but a dedicated nurse at each center provided a detailed explanation of the questionnaire at the time of informed consent before the first cycle of chemotherapy. Furthermore, patients were provided with a diary to use at home, to facilitate documenting the onset and duration (in days) of adverse effects in relation to receiving chemotherapy. However, home diaries were not collected as part of the study material. Treating oncologists were provided with study details (ie, the items listed in the questionnaire) but were not formally involved in the design and execution of the study and were not notified of patients’ participation in the study. The protocol was approved by the ethics committees of the participating institutions. Each patient was required to sign an informed consent before entering the study.
A 10-item paper questionnaire was developed in which the adverse effect definitions and severity grades were translated into Italian from CTCAE version 4.0. The 10 items were nausea, vomiting, constipation, anorexia, dysgeusia, diarrhea, fatigue, pain (generic), paresthesia, and dyspnea. These items were selected from a longer list because of their incidence in this group of patients, their considerable subjective component, and because they usually partially or completely resolve at the time of the next cycle of chemotherapy. Consequently, their reporting is unlikely to be consistent unless the patients are specifically asked.
The presence, grade, and duration of each item was recorded, with duration subdivided into day of onset, duration in days, and persistence at the time of questionnaire administration (ie, 5 fields in total). A sample of the Italian questionnaire and its English translation are available as the eAppendix in the Supplement. Patients were asked to indicate if, after the previous chemotherapy administration, the adverse effect occurred. If the response was “yes” then patients indicated which of a set of statements best described their worst experience of that specific adverse effect, where each statement corresponded to the grades of severity described in CTCAE version 4.0. For example, grade 2 nausea corresponded to, “I felt nauseated and, because of that, I ate and drank less than usual.” Finally, patients were asked to report the day of onset of each adverse effect with respect to the timing of previous chemotherapy (eg, day 3; third day after chemotherapy administration), the duration of adverse effect in days (any severity), and/or whether the adverse effect persisted at the time of completing the questionnaire. When the Italian translation of CTCAE version 4.0 became available and was endorsed by the Italian Association of Medical Oncology in mid-2011,20 the questionnaire was rechecked and no corrections were needed for translational discrepancies.
The dedicated nurse at each center extracted adverse effect information from the medical records of enrolled patients after the first and third adjuvant chemotherapy cycle to complete physician questionnaires. These questionnaires were managed exclusively by nurses and were not available to the treating physicians. Furthermore, the nurses managing physician questionnaires were different to those collecting patient questionnaires at each center.
Percentages and absolute frequencies of symptom grades were calculated for all symptoms reported in patient and physician questionnaires. For each symptom, differences in incidence (grade 0 vs grade ≥1) and severity (mean grade) between the first and second set of questionnaires were assessed using the McNemar test and the Wilcoxon signed rank test. Agreement between patient and physician adverse effect reporting (grade 0 vs grade ≥1) was studied on a per-patient basis using the Cohen κ statistic.21 Both weighted and nonweighted Cohen κ statistics were also computed to assess agreement on each grade (grade 0 vs grade 1 vs grade 2 vs grade 3 vs grade 4). The proportion of pairs for which patients and physicians assigned identical grades was computed, along with the proportion of pairs that disagreed by 1 point for each symptom and the proportion that disagreed by 2 or more points. Finally, for each symptom, the relationship between the number of patients enrolled at each site and the discrepancy in reporting, defined as percentage of physician questionnaires where grade was different from that reported by patient, was evaluated using ordinary least squares regression. Statistical significance was set at P < .05.
A total of 604 women (median age 53.4 years; interquartile range [IQR] 45.0-62.7 years) were enrolled between January 2011 and October 2013 at 11 Italian outpatient oncology clinics. The number of patients enrolled at each site varied between 6 and 236 (eTable 1 in the Supplement). Three patients withdrew consent prior to starting the first cycle of adjuvant chemotherapy. The relevant demographics of the remaining 601 patients are reported in eTable 2 in the Supplement. A total of 596 patient questionnaires were collected after cycle 1 of adjuvant chemotherapy, and 581 patient questionnaires were collected after cycle3. Of these, 594 and 573 had corresponding questionnaires extracted from medical records at the same time point. The median percentage of completed fields was 82% (IQR, 80%-88%) for both cycle 1 and cycle 3 patient questionnaires. The percentage of fields not filled in for each questionnaire item (ie, left blank) is shown in eTable 3 in the Supplement. The percentage of incomplete fields did not exceed 6.28% for 4 of the 5 fields. However, because of the high percentage of incomplete data on persistence or resolution of symptoms at the time of the visit, variables related to symptom duration were not analyzed here.
The results of the 2 patient questionnaires are summarized in Table 1, and cross-tabulated counts are shown in eTable 4 in the Supplement. Overall, there was a reduction in vomiting severity, diarrhea incidence and severity, and pain incidence and severity, and a statistically significant increase in dysgeusia incidence and severity and dyspnea incidence and severity in the second patient-completed questionnaire. With respect to cycle 1 and cycle 3 physician questionnaires, there was an increase in the incidence (2% vs 5%; P = .01) and severity (mean grade, 0.3 vs 0.7; P = .03) of dyspnea (cross-tabulated counts are shown in eTable 5 in the Supplement).
Comparisons between patient and physician questionnaires after the first chemotherapy cycle are summarized in Table 2 (cross-tabulated counts are shown in eTable 6 in the Supplement). The reporting incidence (any grade) was higher in patients than physicians for all adverse-effect items. For example, patients reported constipation, anorexia, and dysgeusia almost twice as frequently as physicians and fatigue 3 times more frequently than physicians. Notably, paresthesia and dyspnea were rarely reported by physicians, but paresthesia was reported by 132 patients (23%) and dyspnea was reported by 142 patients (25%). In fact, 25 patients (4%) reported grade 2 or higher paresthesia and 63 patients reported grade 2 or higher dyspnea (11%). A similar pattern was observed in data collected after the third cycle of chemotherapy (data not shown). Agreement in toxic-effect reporting (grade 0 vs grade ≥1) between patients and physicians for each item after the first chemotherapy cycle is summarized in Table 3. For most items except vomiting, agreement was below the definition of acceptable (Cohen κ statistic, <0.40),22 which was mirrored in the second set of questionnaires (not shown). This low level of agreement might be explained by underreporting by physicians (Table 3). Assessing overall agreement on grade by both weighted and nonweighted Cohen κ statistics and using multiple imputation to account for missing data provided similar results (eTable 7 in the Supplement).
There was also a discrepancy between patients and physicians when grading toxic effects, with patients grading their toxic effects more severely than physicians (eFigure 2 in the Supplement, and Table 2).
Since the number of patients enrolled at each site varied widely (eTable 1 in the Supplement), we sought to assess whether the discrepancies observed between patients and physicians when reporting the grade of toxic effects were associated with the number of patients enrolled. A strong and significant positive association was demonstrated for all 10 items included in the questionnaire (Figure) (eFigure 3 in the Supplement). Similar results were also observed for the second questionnaire (data not shown).
The first aim of this study was to assess whether women undergoing adjuvant chemotherapy for operable breast cancer are able to report and grade 10 common subjective adverse effects using a CTCAE version 4.0–derived questionnaire. Second, owing to the widely recognized problem of underestimating the incidence and severity of chemotherapy-related adverse effects by physicians, patient-derived information was compared with that extracted from their medical records.
There was a very high questionnaire response rate (596 patients [99%] returned a completed cycle 1 questionnaire and 581 patients [97%] returned a completed cycle 3 questionnaire), a high proportion of filled-in fields (median, 82% for both questionnaires), and a low percentage of blank fields with the exception of the question regarding persistence or resolution of toxic effects at the time of chemotherapy visit. For this reason, we decided not to analyze the duration of adverse effects here. However, we are currently analyzing the patient diaries in detail to resolve this issue.
There were significant discrepancies in the reported occurrence and severity of each of the 10 items between patients and physicians. Consequently, there was also low agreement in incidence (grade 0 vs grade ≥1) and grading, mainly owing to underreporting by physicians. Several other authors have reported a similar phenomenon.5-11 For example, Di Maio and colleagues9 recently studied patients enrolled in 3 prospective clinical trials: 1 trial studying elderly patients with breast cancer receiving adjuvant chemotherapy and 2 trials studying patients with advanced non–small-cell lung carcinoma receiving first-line treatment. The authors compared patient and physician reporting of 6 chemotherapy-related adverse effects (anorexia, nausea, vomiting, constipation, diarrhea, and hair loss) after each of the initial 3 cycles of chemotherapy. Physicians used CTCAE v2.0, while patients completed PRO questionnaires issued by the European Organization for Research and Treatment of Cancer (EORTC) that included chemotherapy-related adverse effects rated on a severity scale of “not at all,” “ a little,” “quite a bit,” and “very much.” Agreement between physicians and patients was poor, mainly owing to physicians underestimating both the incidence and severity of adverse effects. Notably, κ values for patient and physician agreement for the 5 adverse effects common to our study (anorexia, nausea, vomiting, constipation, and diarrhea) were remarkably similar to those reported here. These findings suggest that poor physician and patient agreement in adverse-effect reporting is consistent between diverse patient groups (cancer free or metastatic disease), type of treatment received, and management setting (prospective clinical trial or standard treatment).
Our findings also hint at possible reasons for this reproducible phenomenon. First, pairwise comparisons between the 2 patient questionnaires showed trends toward reduced vomiting, diarrhea, and pain. This suggests that physicians may have correctly identified the adverse effects and delivered appropriate management, but because vomiting and diarrhea are common, physicians may not have deemed them sufficiently important to report in the records. Second, there was a statistically significant correlation between discrepancies in adverse-effect reporting and the number of patients enrolled at each institution. Data that we collected during the prestudy surveys provide a possible explanation for this finding (eTable 1 in the Supplement). With 2 exceptions, the number of patients accrued corresponded to enrollment commitment, which had to be estimated on the basis of expected number of patients eligible for the study and allocated resources over a 2-year period. Therefore, we speculate that patient volume and the resources assigned to patient management might be determinants of the discrepancy between physician and patient reporting of adverse effects. In particular, when time is constrained, the physician-patient relationship may focus more on treatment efficacy than on adverse effects, which might be overlooked, not described, or not reported in the medical records. Of note, an Italian study8 examining separate collection of chemotherapy-related adverse effect data by physicians and nurses found greater agreement between patients and nurses than between patients and physicians for most of the items evaluated.8
Our study suffers from limitations common to similar studies in this area, and a randomized study would be difficult to conduct for methodological reasons. Patients were not involved in the development of the Italian version of the questionnaire, and it is possible that a lack of validation of the comprehension of the statements describing toxic effects could be responsible, at least in part, for the observed discrepancies. Physicians at each center were aware of the study but not formally involved. Therefore, they may have consciously or unconsciously focused on the 10 adverse effects considered in this study, thus potentially biasing the discrepancy in adverse effect reporting. Indeed, in a prospective study of an 11-item version of the CTCAE distributed to 435 patients with cancer and their physicians, there was good agreement between physicians and patients for most adverse effects.5 However, for those adverse effects with a subjective component (eg, dyspnea, fatigue), agreement tended to be lower despite the involvement of both patients and physicians.
The strengths of this study are that it is large, prospective, and conducted in a homogeneous population of patients with nonmetastatic breast cancer with exclusion of potential confounders related to baseline cancer symptoms. Most patients were healthy and received anthracycline-based regimens with only minor dose variations at the different participating institutions. Finally, patients received chemotherapy in routine clinical practice and not in a prospective therapeutic trial mandating standardized reporting of adverse effects by the treating oncologist.
We believe that the approach that we tested in this manuscript offers a number of potential benefits to patients with cancer undergoing chemotherapy, in particular those treated in the clinical practice. For example, physician-patient interaction would benefit from using a common reference system for reporting and grading toxic effects. With the diffusion of electronic medical records and the possibility that patients fill in electronic questionnaires, these CTCAE-compliant patient reports could be immediately available to treating physicians to be discussed during the visit.23 Patients can be reassured that physicians adequately weigh their personal experience with adverse effects, which may result in better coping and quality of life during treatment.24 Reconciliation of discrepancies in patient and physician reports during the visit would improve decision making regarding chemotherapy dosing (ie, dose delay or dose reduction).13 This may be particularly effective in high-volume centers, where a constrained physician-patient relationship may be the cause of underreporting of adverse effects by physicians.
This prospective study confirms that patients with breast cancer receiving conventional therapy can provide CTCAE-compliant reporting and grading of common adverse effects related to adjuvant chemotherapy. Similar to other studies, we found that physicians tend to underreport or underestimate adverse effects. However, although we provide some explanation for this phenomenon, there is currently little direct evidence to explain physician-related factors, and further studies are warranted.19
Addressing these open questions is becoming increasingly relevant and in recognition of the importance of considering direct patient involvement in adverse effect collection and grading, the National Cancer Institute is financing, supporting, and disseminating a PRO version of the CTCAE system (PRO-CTCAE)15 based on 78 toxic effect items mapped to the CTCAE. For each toxic effect, PRO-CTCAE contains up to 3 patient questions that define the grade of severity. These questions were developed and validated using a methodologically robust process based on cognitive interviewing techniques.25 The English version of PRO-CTCAE is currently being translated into different languages, including Italian. Our data support the adoption of PRO and highlight the potential of this approach in routine clinical practice.
Corresponding Author: Filippo Montemurro, MD, Unit of Investigative Clinical Oncology, Fondazione del Piemonte per l’Oncologia, Candiolo Cancer Institute IRCCs, Strada Provinciale 142, Pm 3.95, 10060 Candiolo, Italy (email@example.com).
Accepted for Publication: September 26, 2015.
Published Online: December 23, 2015. doi:10.1001/jamaoncol.2015.4720.
Author Contributions: Drs Montemurro and Berchialla had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Montemurro, Culotta, Gallizioli, Grasso, Aglietta, Ballari.
Acquisition, analysis, or interpretation of data: Montemurro, Mittica, Cagnazzo, Longo, Berchialla, Solinas, Culotta, Martinello, Foresto, Calori, Volpone, Bertola, Parola, Tealdi, Giuliano, Ballari.
Drafting of the manuscript: Montemurro, Cagnazzo, Culotta.
Critical revision of the manuscript for important intellectual content: Montemurro, Mittica, Cagnazzo, Longo, Berchialla, Solinas, Culotta, Martinello, Foresto, Gallizioli, Calori, Grasso, Volpone, Bertola, Parola, Tealdi, Giuliano, Aglietta, Ballari.
Statistical analysis: Montemurro, Berchialla.
Obtained funding: Montemurro, Aglietta, Ballari.
Administrative, technical, or material support: Mittica, Cagnazzo, Longo, Solinas, Martinello, Foresto, Gallizioli, Calori, Grasso, Volpone, Bertola, Parola, Tealdi, Giuliano, Ballari.
Study supervision: Montemurro, Mittica, Cagnazzo, Culotta, Gallizioli, Aglietta.
Conflict of Interest Disclosures: Dr Montemurro served as a consultant for GlaxoSmithKline and as a member Speaker’s bureau for Astra Zeneca and Hoffmann La Roche. No other disclosures are reported.
Funding/Support: This work was supportedd by Rete Oncologica Del Piemonte e della Valle d'Aosta and Fondazione Piemontese per la Ricerca sul Cancro (FPRC onlus) fondi 5 per mille 2012 Ministero della Salute.
Role of the Funder/Sponsor: The funders/sponsors had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Additional Information: The results of this paper have been partially presented as a poster at the annual meeting of the American Society of Clinical Oncology; May 30, 2015; Chicago, IL.