Effect of the Serious Illness Care Program in Outpatient Oncology: A Cluster Randomized Clinical Trial

Key Points

Question  Does a quality-improvement intervention designed to enhance communication between oncology clinicians and patients with advanced cancer improve goal-concordant care, peacefulness, therapeutic alliance, anxiety, and depression?

Findings  In this cluster randomized clinical trial evaluating use of a communication quality-improvement intervention by 91 clinicians for 278 patients with advanced cancer, results were null with respect to the coprimary outcomes of goal-concordant care and peacefulness for decedents, but there were fewer than expected patients with evaluable coprimary outcomes. However, the trial demonstrated significant reductions in moderate to severe anxiety and depression symptoms among intervention patients in the total population.

Meaning  This cluster randomized clinical trial highlighted the challenges of measuring outcomes near the end of life among patients with advanced cancer; secondary outcomes of anxiety and depression are important and require further study.

Importance  High-quality conversations between clinicians and seriously ill patients about values and goals are associated with improved outcomes but occur infrequently.

Objective  To examine feasibility, acceptability, and effect of a communication quality-improvement intervention (Serious Illness Care Program) on patient outcomes.

Design, Setting, and Participants  A cluster randomized clinical trial of the Serious Illness Care Program in an outpatient oncology setting was conducted. Patients with advanced cancer (n = 278) and oncology clinicians (n = 91) participated between September 1, 2012, and June 30, 2016. Data analysis was performed from September 1, 2016, to December 27, 2018. All analyses were conducted based on intention to treat.

Interventions  Tools, training, and system changes.

Main Outcomes and Measures  The coprimary outcomes included goal-concordant care (Life Priorities) and peacefulness (Peace, Equanimity, and Acceptance in the Cancer Experience questionnaire) at the end of life. Secondary outcomes included therapeutic alliance (Human Connection Scale), anxiety (Generalized Anxiety Disorder 7 scale), depression (Patient Health Questionnaire 9), and survival. Uptake and effectiveness of clinician training, clinician use of the conversation tool, and conversation duration were evaluated.

Results  Data from 91 clinicians in 41 clusters (72.9% participation; intervention, n = 48; control, n = 43; 52 [57.1%] women) and 278 patients (45.8% participation; intervention, n = 134; control, n = 144; 148 [53.2%] women) were analyzed. Forty-seven clinicians (97.9%) rated the training as effective (mean [SD] score, 4.3 [0.7] of 5.0 possible); of 39 who received a reminder, 34 (87.2%) completed at least 1 conversation (median duration, 19 minutes; range, 5-70). Peacefulness, therapeutic alliance, anxiety, and depression did not differ at baseline. The coprimary outcomes were evaluated in 64 patients; no significant differences were found between the intervention and control groups. However, the trial demonstrated significant reductions in the proportion of patients with moderate to severe anxiety (10.2% vs 5.0%; P = .05) and depression symptoms (20.8% vs 10.6%; P = .04) in the intervention group at 14 weeks after baseline. Anxiety reduction was sustained at 24 weeks (10.4% vs 4.2%; P = .02), but depression reduction was not sustained (17.8% vs 12.5%; P = .31). Survival and therapeutic alliance did not differ between groups.

Conclusions and Relevance  The results of this cluster randomized clinical trial were null with respect to the coprimary outcomes of goal-concordant care and peacefulness at the end of life. Methodologic challenges for the primary outcomes, including measure selection and sample size, limit the conclusions that can be drawn from the study. However, the significant reductions in anxiety and depression in the intervention group are clinically meaningful and require further study.

Trial Registration  ClinicalTrials.gov identifier: NCT01786811

In their last year of life, patients with serious illness experience physical and emotional distress, inadequate communication with clinicians, and medical interventions inconsistent with patient priorities and preferences.^1-7 Patients who discuss end-of-life care with their clinicians, especially earlier in the disease trajectory, are more likely to have positive outcomes, including better quality of life, less distress, and a higher likelihood of receiving care consistent with their preferences.^8-10 However, evidence indicates gaps in the frequency, timing, and quality of such conversations.^11-14 To address these deficiencies, national medical organizations have called for improved communication about patients’ values, goals, and care preferences (serious illness conversations).^15,16

Although palliative care clinicians train for this task, a limited palliative-care workforce suggests the need for other clinicians to effectively lead serious illness conversations.^17-20 However, interventions seeking to equip non–palliative-care specialists to better communicate with patients about end-of-life concerns have not improved patient outcomes, such as psychological symptoms or quality of life.^21-25 In one trial of trainee physicians, a communication training program was associated with an increase in patients’ depressive symptoms, raising concerns that end-of-life conversations may worsen psychological symptoms.²⁵ In addition, clinicians cite concerns about harming patients as a barrier to initiating these conversations.^26,27 Because non–palliative-care clinicians must fill this gap in communication with patients, we systematically developed and extensively pilot tested the Serious Illness Care Program (SICP) with clinicians and patients.²⁸

The present trial evaluated the feasibility and acceptability of our intervention (SICP), including uptake and effectiveness of training, adoption of the conversation guide, and duration of conversations, and its effect on patient outcomes, including goal-concordant care and peacefulness at the end of life (coprimary outcomes) and therapeutic alliance, anxiety, depression, and survival for the total population (secondary outcomes). We chose the measures of peacefulness and goal-concordant care because they are patient centered, important to patients and caregivers, and do not make assumptions about patients’ care preferences (life prolonging vs comfort focused).

We conducted a cluster randomized clinical trial from September 1, 2012, to June 30, 2016, at the Dana-Farber Cancer Institute, a National Cancer Institute–designated cancer center, in Boston, Massachusetts. The Dana-Farber Cancer Institute Institutional Review Board reviewed and approved the study. The protocol is available in Supplement 1. All clinicians and patients provided written informed consent. Each clinician received a $150 gift card for participation; patients and caregivers received no compensation.

We stratified clinician clusters by disease center or satellite facility, and within strata, randomized approximately one-half of the clusters to the intervention (n = 20) and one-half to control (n = 21). Enrolled clinicians were not blinded to study arm; patients were blinded to the study arm of their clinicians.

Power calculations were performed for the study’s primary outcomes. To ensure an overall 5% type I error rate, we used a 2.5% type I error rate for each of the 2 primary hypotheses. We based the power calculations on having 200 evaluable patients per study arm and allowed for 6% unevaluability owing to patient dropout.

The intervention included tools, training, and system changes.²⁸ Clinical tools included a Serious Illness Conversation Guide (SICG) for clinicians,¹⁴ a patient letter introducing the SICG, and a Family Guide after the discussion.²⁸ Clinician training included a 2.5-hour interactive, skills-based training session on the SICG delivered by palliative care experts who offered follow-up coaching. System changes included routine identification of patients at high risk of death, email reminders to initiate conversations (reminders), and a novel structured template in the electronic medical record for SICG documentation. Control clinicians provided usual care; control patients did not receive supporting tools.

We invited clinicians (physicians, physician assistants, and nurse practitioners) from 10 disease centers and 2 satellite clinics to enroll. We excluded gynecology-oncology clinicians (participating in a concurrent study on end-of-life care) and melanoma clinicians (pilot subjects). We defined clusters as units of clinicians within a disease center based on clinical workflow; a typical cluster included 1 nurse practitioner or physician assistant and 2 to 3 physicians. Cluster sizes varied. Enrolled oncology clinicians identified eligible patients by reviewing patient lists at regular intervals and answering the question about surprise, “Would I be surprised if this patient died in the next year?”²⁹ Patients for whom clinicians responded no were eligible for participation. We excluded patients with cognitive impairment, those who did not speak English, and patients unable to identify a caregiver. Participation rates for clinicians or patients were calculated separately as the number who consented and enrolled divided by the total number invited to participate.

All patients completed a baseline survey at enrollment and follow-up surveys approximately every 2 months for 2 years or until death. Lacking a criterion standard for measuring the coprimary outcome of concordance between patient goals and care provided at the end of life,³⁰ 2 novel, patient-centered tools that did not prejudge patient values were developed: Life Priorities survey for patients and the Family Perceptions survey.²⁸ Based on a list of 9 common goals drawn from an extensive literature review and patient interviews, we asked patients to select and rank 5 goals in order of importance and, following patient death, we asked caregivers whether each goal was fulfilled in the patient’s final week and final 3 months of life.^7,9,31-34

We assessed the coprimary outcome of peacefulness in decedents through the Peace, Equanimity, and Acceptance in the Cancer Experience (PEACE) questionnaire, a validated tool yielding 2 subscales: Struggle with Illness, measuring feelings of upset, worry, unfairness, shame, and anger at diagnosis (7 questions; score range, 7-28; Cronbach α = .81); and Peaceful Acceptance, measuring acceptance of diagnosis, inner calm, and feelings of being well-loved (5 questions; score range, 5-20; Cronbach α = .78.)³⁵

We measured therapeutic alliance (secondary outcome) with a modified version of The Human Connection (THC) scale, which evaluates patients’ sense of mutual understanding, caring, and trust with their physicians.³⁶ To decrease patient burden and avoid redundancy, we included 7 of the original 16 items (Cronbach α = .90), a reduction supported by the tool developer (J. W. Mack, MD, PhD, and R. E. Bernacki, MD, MS, in-person communication, May 7, 2012). Scores on this shortened THC scale range from 7 (lower therapeutic alliance) to 28 (higher therapeutic alliance) (Cronbach α = .83 in these trial data). We assessed anxiety and depressive symptoms (secondary outcomes) using the Generalized Anxiety Disorder-7 Scale and Patient Health Questionnaire 9, respectively. The Generalized Anxiety Disorder 7 Scale evaluates symptoms according to the criteria of the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (7 items, range 0-21). Scores in the moderate or severe category (≥10) were considered clinically significant (Cronbach α = .92). The Patient Health Questionnaire 9 evaluates symptoms of major depressive disorder according to the criteria of the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (9 items, range 0-27). Scores in the moderate or severe category (≥10) were considered clinically significant (Cronbach α = .86 to .89).^37,38 We identified patient deaths from the Dana-Farber Clinical Operational and Research Information System database.²⁸

At baseline, we surveyed clinicians in both arms about their profession (physician, physician assistant, nurse practitioner), sex, years in practice, percentage clinical time, and disease center. After training, we surveyed clinicians for training effectiveness (Likert scale range, 0 [not at all effective]-5 [very effective]). After sending a reminder, we surveyed clinicians for conversation occurrence and duration.

We used proportions for categorical variables and means or medians for continuous variables. All comparisons across study arms accounted for clustering of patients within clinician teams. We considered a 2-tailed P value ≤.05 as statistically significant. All analyses were conducted based on intention to treat.

When comparing baseline clinician and patient characteristics between arms, we used generalized estimating equations,³⁹ χ² tests for categorical variables, and t tests for continuous variables.

We evaluated goal-concordant care by matching each decedent’s final Life Priorities²⁸ survey (within 3 months of death) with their caregiver’s Family Perceptions²⁸ survey. We scored each of the patient’s 3 highest ranking goals as concordant if the caregiver indicated the goal had been achieved to a large extent, resulting in a score of 0, 1, 2, or 3 goals met. We compared the arms using a generalized estimating equation score test similar to Wilcoxon rank sum test for ordinal categorical data.³⁹ Using generalized estimating equation χ² tests for ordinal data, we compared both PEACE³⁵ subscales for decedents at baseline and at 3 months before death across the study arms.

We created a separate model for each outcome of interest, using a continuous score for therapeutic alliance and dichotomizing anxiety³⁷ and depression³⁸ as moderate or severe vs none or mild in the total patient population. Owing to variation in timing (patients did not complete surveys at the same fixed points), we fit repeated-measures models via generalized estimating equations.^39,40 We calculated the mean therapeutic alliance score and logits of the probabilities of moderate or severe anxiety and moderate or severe depression as a cubic spline of time of survey, using all data on all patients from all time points in an intention-to-treat, repeated-measures model.⁴¹ We performed autoregressive modeling to correlate outcome measures for the same patient at 2 different time points.⁴² Because conversations occurred, on average, 12 weeks after baseline in the intervention arm, we compared patient outcomes across study arms at 14 and 24 weeks after baseline (the average completion time for the next 2 surveys), using the estimated means and probabilities at these 2 times from the repeated-measures models.

We fit separate spline models for control and intervention arms, allowing the curves to vary over time differently in each arm, and used inverse propensity weighting to balance the 3 outcomes between the 2 arms at baseline to ensure that differences at later time points were not due to baseline differences, even though differences were nonsignificant at baseline.^43,44 For each outcome, we modeled the propensity score (probability of being in the intervention arm) via logistic regression with baseline outcome (therapeutic alliance scores, anxiety, or depression), and patient characteristics as predictors. Although dropout and survival did not differ between arms, the models⁴⁰ protected against potential biases arising from patients in one arm being followed up for longer periods. We obtained Kaplan-Meier 2-year survival estimates from date of baseline and used a log-rank test to compare survival differences between all enrolled intervention and control patients.

Data analysis was performed from September 1, 2016, to December 27, 2018. Statistical analyses were performed with SAS software, version 9.4 (SAS Institute Inc).

A total of 91 oncology clinicians, grouped into 41 randomized clusters (72.9% participation; intervention, n = 48; control, n = 43; 52 [57.1%] women), were enrolled. A total of 379 patients were enrolled, 278 of whom had analyzable data (45.8% participation; intervention, n = 134; control, n = 144; 148 [53.2%] women). The number of clusters decreased from 41 (number of clinician clusters) to 35 (number of clusters for analyzable patients) because some clinician clusters either enrolled no patients or their patients’ data were not able to be analyzed (Figure 1). A total of 209 patients (75.2%) completed at least 1 postbaseline survey. Compared with patients who participated, patients who did not participate were significantly older (mean age, 62.0; 95% CI, 59.7-64.2 years vs 65.6; 95% CI, 63.1-68.2 years; P < .001) and less likely to have breast cancer (97 [27.6%] vs 64 [17.3%]; P = .04), although there were no sex differences. Patients with analyzable vs nonanalyzable data were significantly more likely to be married (222 [79.9%] vs 38 [55.9%]; P = .002) and have higher incomes (P = .03) than those with nonanalyzable data; no other demographic differences were significant. Neither baseline clinician (Table 1) nor baseline patient (Table 2) characteristics demonstrated significant differences between arms.

We trained 47 of 48 (97.9%) intervention clinicians, and clinicians rated the training as effective (mean [SD] score, 4.3 [0.7] of 5.0 possible). Of those trained, 39 clinicians (83.0%) received at least 1 reminder to conduct a serious illness conversation, and of those reminded, 34 (87.2%) completed at least 1 conversation. Clinicians reported a median conversation duration of 19 minutes (range, 5-70).

For decedents, we matched a Family Perception²⁸ survey to an appropriately timed Life Priorities²⁸ survey for 64 decedents (38 intervention, 26 control). Erroneously, the Life Priorities goal “not be a burden” was not included in the Family Perception questionnaire. For the purpose of this analysis, patients who selected this goal as 1 of their top 3 goals had this goal replaced by the subsequent goal. The sensitivity analysis omitted the patients for which this applied (n = 18). There was no significant difference in the median number of top 3 goals met between study arms. There were no significant differences between study arms for decedents in the PEACE³⁵ subscales at baseline or within 3 months before death (Table 3).

Among all patients, mean scores of THC scale³⁶ did not differ significantly between arms at baseline (intervention: 25.3; 95% CI, 24.8-25.8 vs control: 25.5; 95% CI, 25.0-26.0; P = .60), at 14 weeks after baseline (intervention: 25.5; 95% CI, 24.8-26.2 vs control: 25.7; 95% CI, 25.1-26.2; P = .65), or at 24 weeks after baseline (intervention: 25.5; 95% CI, 25.0-26.1 and control: 25.4; 95% CI, 24.8-26.0; P = .71) (Figure 2A).

The proportion of patients in the total population reporting moderate or severe anxiety symptoms did not differ significantly between arms at baseline (9.6% control vs 9.0% intervention; P = .85). At 14 weeks after baseline, the proportion of patients reporting moderate or severe anxiety symptoms was significantly lower in the intervention arm (10.2% vs 5.0%; P = .05). At 24 weeks after baseline, intervention patients remained less likely than control patients to report moderate or severe anxiety symptoms (10.4% vs 4.2%; P = .02) (Figure 2B).

Among all patients, the proportion of patients reporting moderate or severe depression symptoms did not differ significantly between arms at baseline (20.4% control vs 19.3% intervention; P = .84). At 14 weeks after baseline, the proportion of patients reporting moderate or severe depression symptoms was significantly lower in the intervention arm (20.8% vs 10.6%; P = .04). At 24 weeks after baseline, the proportion of patients reporting moderate or severe depression symptoms did not differ significantly between arms (17.8% vs 12.5%; P = .31) (Figure 2C).

Median 2-year survival did not differ between study arms (intervention, 13.9 months; control, 13.6 months; log-rank P = .91) (eFigure in Supplement 2).

The results of this cluster randomized clinical trial of a communication quality-improvement intervention were null with respect to the coprimary outcomes of goal-concordant care and peacefulness at the end of life and the secondary outcome of therapeutic alliance. However, the trial demonstrated a significant improvement in depression symptoms and a significant and sustained improvement in anxiety symptoms in intervention patients. Survival did not differ significantly between arms.

Several explanations for the lack of effect of our intervention on the primary outcomes are possible. First, a smaller number of expected deaths and poor survey response limited our sample size for the coprimary outcomes of peacefulness and goal-concordant care. Because the point estimates for the differences between the arms had wide 95% CIs that included zero and did not represent clinically important differences, we are unable to conclude whether any meaningful benefit or harm resulted from the intervention. Second, because of the absence of a strong patient-centered measure of goal-concordant care, we used an unvalidated survey. Third, our measurements of goal-concordant care were dependent on patient responses late in the illness and family responses early in bereavement, which may have been too burdensome.⁴⁶ Fourth, although we measured peacefulness with a validated scale,³⁵ this measure may have been inadequate to capture elements of peacefulness that respond to improved communication.

As a result, we are uncertain whether our intervention was ineffective at improving these outcomes, if our outcome measures were not appropriate or feasible, or if we lacked sufficient numbers to detect meaningful differences. Our challenges reflect the need in our field for patient-centered measures of communication that are agreed upon, validated, and demonstrably sensitive to communication interventions.^47-51

This trial demonstrated significant improvements in the secondary outcomes of moderate to severe anxiety and depression symptoms that regularly occur in patients with cancer.^52-54 In contrast to prior research,^24,25,55 this study, using well-validated and widely used measures, demonstrated significantly decreased rates of anxiety and depression symptoms within 2 weeks of the conversation in the intervention group, and the reduction in anxiety symptoms lasted until at least 24 weeks after baseline, suggesting that trained oncology clinicians can discuss important and difficult topics without causing harm and with potential benefit. To our knowledge, this is the first study to identify a clinically meaningful benefit to psychological symptoms from a structured communication approach, suggesting that psychological outcomes be considered primary outcomes in future communication studies in oncology. This finding also highlights the need for measurement of communication and outcomes over the illness trajectory—not just at the end of life—which may help clinicians to better understand how to improve patients’ well-being as they live with serious illness.

We found that intervention clinicians readily adopted the program; they attended the training session and rated it as effective. They then conducted serious illness conversations in a feasible time frame with respect to the constraints of a typical oncology practice. In a separate article,⁵⁶ we report higher quality and more frequent conversations documented earlier in the intervention group. We expect these findings to be transferrable to other clinical contexts that treat patients with advanced cancer while also recognizing that these intervention components require substantial organizational resources.

Among several study limitations was a significantly smaller sample size than expected for our primary outcomes. Although low, the patient participation rate in the trial is consistent with that of other population-level trials of seriously ill patients.^46,57 Because of lower patient accrual rates, fewer deaths than expected, longitudinal design, and difficulties obtaining surveys from patients and bereaved caregivers, a relatively large number of patients were not included in the primary outcomes analysis. However, nonparticipants and unanalyzed participants were not meaningfully different from those who were analyzed, and randomization still produced comparable groups between study arms.

The variation in timing of outcome assessment may also be a limitation; however, we found that dropout and timing of measurement were similar across arms. Use of the surprise question by all clinicians and frequent survey completion by all patients may have prompted conversations in the control arm, attenuating potential between-arm differences. In addition, findings may not be generalizable because the study was conducted at a single oncology institution with a fairly small number of participants, most of whom were white, college educated, and affluent. Another limitation is that the multicomponent nature of the intervention prevents assessment of which components contributed to the outcomes.

The results of this cluster randomized clinical trial were null with respect to the coprimary outcomes of goal-concordant care and peacefulness for decedents, but were limited by a small sample size for the primary outcomes. However, the findings demonstrated significant reductions in the secondary outcomes of anxiety and depression symptoms immediately after the conversation and a sustained reduction in anxiety among intervention patients in the total population. This study showed that a feasible system-level communication intervention with high acceptance by clinicians may improve some meaningful patient outcomes among those with advanced cancer. Further development of serious illness communication interventions will require more reliable and well-accepted patient-centered outcome measures and additional testing of the effect on patients throughout their illness trajectory.

Accepted for Publication: January 11, 2019.

Corresponding Author: Rachelle Bernacki, MD, MS, Ariadne Labs, Brigham and Women’s Hospital, Harvard T. H. Chan School of Public Health, 401 Park Dr, 3rd Floor E, Boston, MA 02215 (rbernacki@ariadnelabs.org).

Published Online: March 14, 2019. doi:10.1001/jamainternmed.2019.0077

Author Contributions: Drs Bernacki and Paladino are co–first authors, contributed equally to the study, had full access to all of the data in the study, and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Bernacki, Paladino, Hutchings, Kavanagh, Miller, Lipsitz, Gawande, Block.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Bernacki, Paladino, Kavanagh, Geerse, Miller, Lipsitz.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Paladino, Neville, Geerse, Miller, Lipsitz.

Obtained funding: Bernacki, Gawande, Block.

Administrative, technical, or material support: Bernacki, Paladino, Hutchings, Kavanagh, Lakin, Block.

Supervision: Bernacki, Paladino, Lipsitz, Gawande, Block.

Conflict of Interest Disclosures: Dr Gawande received compensation from writing and other media on health care, including the subject of serious illness care and checklists, during the conduct of the study, and is employed by the new nonprofit-seeking health care venture parented by Amazon, Berkshire Hathaway, and JP Morgan Chase. Dr Block receives compensation from Up to Date Palliative Care Editor. No other disclosures were reported.

Funding/Support: The trial was supported by the Branta Foundation, Charina Endowment Fund, Margaret T. Morris Foundation, Richard A. Cantor Fund, Partners Healthcare, and the John A. Hartford Foundation.

Role of the Funder/Sponsor: The funding organizations had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: The content is solely the responsibility of the authors and does not necessarily represent the official views of the Branta Foundation, Charina Endowment Fund, Margaret T. Morris Foundation, Richard A. Cantor Fund, Partners HealthCare, and the John A. Hartford Foundation.

Meeting Presentation: This paper was presented at the American Academy of Hospice and Palliative Medicine 2019 Annual Assembly; March 14, 2019; Orlando, Florida.

Additional Contributions: Samantha Epstein, MD, Jenna Ogden, BS, Nathan Persch, BA, Katrinka Quirk, MS, Grant Smith, MD, Maribel Valenzuela, MS, Judith Vick, MPH, Alyssa Vigliotti, MD, and Kathleen Yates, BS, contributed to data collection; Meghna Desai contributed to editing; Ami Karlage, BA (Ariadne Labs), contributed to critical revision of the manuscript content. There was no financial compensation. We thank the patients and clinicians who participated in this study.

Data Sharing Statement: See Supplement 3.