Efficacy of a Communication-Priming Intervention on Documented Goals-of-Care Discussions in Hospitalized Patients With Serious Illness

Key Points Question Among hospitalized adults with serious illness, does a patient-specific communication-priming intervention (Jumpstart) targeting both patients and their inpatient clinicians increase documented goals-of-care discussions compared with usual care? Findings In this 2-hospital randomized clinical trial of 150 patients, the Jumpstart intervention resulted in a significant increase in electronic health record–documented goals-of-care discussions between randomization and hospital discharge (8% of patients in the usual care group vs 21% in the intervention group). Patient-reported or surrogate-reported goals-of-care discussions did not differ between groups. Meaning These findings suggest that a communication-priming intervention may be considered in the inpatient setting to increase goals-of-care discussions.

People near the end of life often receive care they would not choose. 13,14 A recent report from the 98 Institute of Medicine documents these discrepancies in care and identifies advance care planning and 99 goals-of-care discussions as a primary mechanism for addressing them. 13 This type of 100 communication is a focus for improvement for two key reasons: 1) clinicians frequently do not have 101 goals-of-care discussions with their patients until very late in the illness; [1][2][3][4][5] and 2) when these 102 discussions occur, they are associated with improved quality of care and patient-and family-centered 103 outcomes including increased quality of life and fewer intensive treatments at the end of life. 1,15-17 104 Goals-of-care discussion should start in the outpatient setting when patients are well enough to 105 participate, in order to inform "in the moment" clinical decisions. 18,19 For hospitalized patients with 106 chronic illness, a key component of high quality care includes goals-of-care discussions conducted 107 early during a hospital stay that build upon prior discussions and identify how patients' goals of care 108 should inform current care plans. 3,19,20 These early hospital discussions are also supported by the 109 National Quality Forum (NQF). 21 However, despite their key importance to a large number of 110 patients, these hospital goals-of-care discussions often do not occur. 3,22 111 The recent research agenda in Annals of Internal Medicine for serious illness communication 112 highlights the importance of promoting high-quality goals-of-care discussions, as well as the potential 113 opportunity to use the EHR to both identify those patients who would benefit from goals-of-care 114 discussions and to guide clinicians in high-quality discussions. 6 We are conducting a pilot trial to 115 examine the efficacy of such an intervention and facilitate the development and funding of an 116 innovative hybrid effectiveness/implementation trial that evaluates the intervention and its 117 implementation. 23 118 119 b. Innovation 120 121 Use of the EHR to identify seriously ill, hospitalized patients without a goals-of-care discussion: 122 We will use our EHR-based quality metrics program to identify hospitalized patients with chronic life-123 limiting illness or age >80 who do not have EHR documentation of a goals-of-care discussion, thereby 124 targeting a population likely to benefit from the intervention. We proposed using an innovative 125 NLP/ML protocol to identify inpatient and outpatient documentation of goals-of-care discussions; our 126 preliminary data suggested an average accuracy of 90.1% for identifying this documentation, based 127 on a dataset of 722 verified positive goals-of-care notes and 1671 negative notes. However, for this 128 trial we opted to use the gold standard of manual abstraction to identify goals-of-care discussions.

129
Deliver a bilateral communication-priming intervention for goals-of-care discussions in the hospital 130 setting: The intervention is based on our recently completed randomized trial of the Jumpstart 131 intervention: a patient-specific pre-conversation communication-priming intervention targeting both 132 patients and clinicians by providing each with information obtained from patient-reported surveys that 133 is used to guide a goals-of-care discussion. In an outpatient study of 537 patients, the intervention 134 increased goals-of-care discussions from 31% in the control group to 75% in the intervention 135 (p<0.001) and increased patient-assessed quality of communication (p<0.001). 2,12 For this pilot trial, 136 we adapted the intervention for hospitalized patients. 137 Develop an innovative effectiveness/implementation trial that advances implementation science in 138 palliative care: This pilot assesses the feasibility, acceptability, and implementation of its methods 139 and outcomes in the inpatient setting. We will follow this pilot with a novel hybrid 140 effectiveness/implementation trial that would accelerate dissemination of the intervention by allowing 141 us to evaluate implementation strategies and outcomes that may facilitate uptake of the 142 intervention. [23][24][25][26][27] This innovative design offers the opportunity to advance implementation science in 143 palliative care, increasing the utility and fundability of the next grant. i. Overview: We will conduct a pilot randomized trial of an intervention to promote and guide goals-of-149 care discussions for seriously ill hospitalized patients using an automated method for identifying 150 eligible participants. The trial will assess feasibility and acceptability (Aim 1) as well as efficacy for 151 prompting discussions (Aim 2) and will use qualitative methods to explore barriers and facilitators to 152 implementation and opportunities to improve the intervention (Aim 3). 153 154 ii. Setting: We will conduct this study at the two largest hospitals in the UW Medicine system. The 155 University criteria, we will include only those with no identified documentation of goals-of-care discussions during 171 the current hospitalization, as determined through daily screening of hospitalized patients using 172 methods developed by our palliative care metrics program. 7-10 Patients will be eligible after a 12 hour 173 stay with no maximum stay. 174 175 iv. Study design: In this pilot randomized trial, eligible patients will be assigned to intervention or 176 usual care in a 1:1 ratio using variable size blocks and stratifying randomization by hospital. 177 178 v. Intervention: The intervention has four components ( Figure 1). First, we will use our metrics 179 program to identify seriously ill hospitalized patients. Second, consented patients will complete a 180 survey assessing three domains: a) preferences for discussions about goals of care; b) most 181 important barrier and facilitator for having such 182 discussions; and c) current goals of care. If 183 patients are not able to complete a survey, we 184 will recruit a legal surrogate decision-maker to 185 participate and complete the survey. We will 186 use the protocol from our recent randomized trial 187 to create a "Jumpstart form" to prompt and guide 188 a goals-of-care discussion between the patient 189 and physician team caring for the patient or, if 190 the patient isn't able, the family member and the 191 physician team (including physicians, nurse 192 practitioners, and physician assistants). Third, 193 we will use our NLP/ML approach to identify 194 goals-of-care discussions, POLST forms, or advance directives in the UW Medicine EHR prior to this 195 admission (inpatient and outpatient) and include this information on the Jumpstart forms. Fourth, we 196 will deliver the Jumpstart form to the primary physician team (all attending and resident physicians, 197 subinterns, and advance practice providers on the primary team caring for the patient) via secure 198 email with in-person delivery when possible, and we will also provide the patient or family with a 199 patient/family version of the form. The Jumpstart forms will be delivered within 1-2 business days of 200 randomization, as supported by the NQF. 21 The forms provide a distilled version of the patient/family 201 survey responses and, based on the responses, patient-specific suggestions for conducting goals-of-202 care discussions with this patient or family. The suggestions will be guided by the experience and 203 training of VitalTalk and adapted to the inpatient setting. 37,38 All forms include a link to a 3-minute "just-204 in-time" training video by VitalTalk on using the form (tailored to clinicians, patients, or family). 205 206 vi. Comparison group: The comparison group will receive usual care plus surveys, without steps 3 and 207 4. 208 209 vii. Major outcomes: Aim 1 outcomes assessing the feasibility and acceptability of the intervention will 210 be evaluated with the completion of study activities by those randomized to the intervention. . Aim 2 211 outcomes will be assessed with patient/family-reported surveys completed by both intervention and 212 comparison groups. 213 Feasibility and acceptability (Aim 1): Feasibility will be measured with the following: a) survey 214 completion rates with the expectation that successful feasibility will be supported with >80% 215 completion of patient/family surveys; b) receipt of the Jumpstart form with the expectation that >80% 216 of clinicians and patients/families indicate having received the form; and c) use of the form, with the 217 expectation that >80% of clinicians and patients/families will report that they have read the form. We 218 will also add an open-ended question for participants to provide suggestions for improving the 219 intervention.

220
EHR documentation of goals-of-care discussions (Aim 2): Documentation of goals-of-care 221 discussions will be evaluated using both manual chart review and our NLP/ML methods, with manual 222 chart review using our standard EHR abstraction methods providing the gold standard. 39 having more pain and discomfort, or would you want a plan of care that focuses on relieving pain and 251 discomfort as much as possible, even if that means not living as long?" The next question assesses 252 patients' perceptions of current treatment using the same two options. 63  prior studies, 63 we expect only 60% of patients will be receiving care concordant with their goals. 2 259 Implementation: We will collect qualitative data on barriers and facilitators for implementation, 260 guided by the Consolidated Framework for Implementation Research, including those related to the 261 intervention, settings (inner and outer), processes, and individuals (see Aim 3 analyses). 26 262 263 ix. Description of participants: For all participants, we will collect age, gender, race/ethnicity, and 264 education (or profession for clinicians). For patients, we will collect comorbidities. 67 For family, we will 265 collect relationship with the patient. 266 267 x. Quantitative data collection 268 Surveys: Surveys will be completed by patients/family at two points in time:1) at enrollment; and 2) 269 at 4-5 business days after randomization. Clinicians will complete surveys at time point 2. Surveys 270 may be completed in-person, online, or by phone, based on respondents' preferences.

271
Patients and family members: Patients will be surveyed if they are able. Study staff will use a 272 brief six-item screening tool to assess cognitive impairment. 68 If patients are not able to participate, 273 we will identify a legal surrogate decision-maker to consent for both the patient and themselves. The prior studies, we found some patients in the comparison group who had not received the form 280 mistakenly thought that they received it and, therefore, these are important data to collect. 281 Clinicians: At patient enrollment, we will collect data on clinicians on the acute care team from 282 hospital records (e.g., age, gender, specialty, level of training) in both arms. After the intervention, a 283 "primary clinician" will be identified for intervention patients (the clinician who did or could have had a 284 goals-of-care discussion) and asked items assessing the intervention: 1) Did he/she complete a goals 285 of care discussions with patient and/or family? 2) If not, what were the reasons for not having had this 286 discussion? 3) Was the Jumpstart used?3) Would he/she recommend the Jumpstart to other 287 clinicians? 288 289 EHR: We will use our EHR-based quality metrics program to obtain data about patients from the 290 EHR, 7-10,69 collecting demographics, co-morbidities, and documentation of goals-of-care discussions 291 preceding and during the patient's inpatient stay. In addition, we will conduct a manual chart review to 292 corroborate the documentation of goals-of-care discussions using study staff trained to identify these 293 discussions. 2,39-41 294 295 Qualitative data collection: Aim 3 will use data from 30 semi-structured interviews with patients, 296 family members and clinicians. We will use purposive sampling to ensure a diverse group based on 297 race/ethnicity, age, gender, and, for clinicians, specialty and year of training, Participants will be 298 interviewed by a trained qualitative researcher using an interview guide and interviews will be audio-299 recorded and transcribed, similar to our prior qualitative research. 70-77 300 301 302 d. Aim-specific Analyses: 303 304 Aim 1: Pilot randomized trial to evaluate the intervention's feasibility and acceptability. 305 We will assess for successful implementation of the intervention using descriptive statistics to 306 examine the proportion of eligible patients who are enrolled and, among those randomized to the 307 intervention, the proportion of clinicians and patients/families who receive the intervention. We 308 anticipate that 80% of patients randomized to the intervention will receive the intervention and, with 309 our sample size (n=75 intervention patients), we will be able to identify this proportion with 95% 310 confidence intervals of ±7%. We will also examine feasibility of the intervention components as the 311 proportion of enrolled patients for whom each one of the four intervention steps are successfully 312 completed (see Figure 1).

314
Aim 2: Evaluate the efficacy of the intervention for changing processes of care. 315 The primary outcome of this trial is the documentation of a goals-of-care discussion in the EHR, 316 which will be assessed with a logistic regression model with adjustment for hospital site and actual 317 confounders. Actual confounders will be patient characteristics (listed above) that change the 318 coefficient for the relationship between intervention-control predictor and the outcome by more than 319 10%. We will adjust for confounders in this pilot trial in order to maximize the accuracy of the 320 treatment estimate. 78 The intervention's effect on the quality of communication about goals-of-care 321 will be assessed with a composite QOC_eol outcome, collected from patients or family members. The 322 test will use a linear regression model, estimating the coefficient for the outcome regressed on the 323 control/intervention predictor, after adjustment for actual confounders (as above). 324 We will collect data on the other outcomes to ensure feasibility of outcome assessment with this 325 study design. We will perform descriptive statistics for all outcomes to understand distribution, range, 326 and central tendencies, but we will not report hypothesis testing for these variables in this pilot. 79-81 327 328 Aim 3: Interviews to identify barriers and facilitators for implementation of the intervention. 329 We will perform a modified grounded theory analysis of transcribed interviews to explore feedback 330 on the intervention, ways to improve the intervention delivery and implementation, and aspects of care 331 not adequately addressed by the intervention. 82-85 Interview guides and analyses will be guided by the 332 Consolidated Framework for Implementation Research to explore factors affecting implementation, 333 within 5 domains: intervention, settings, processes, individuals 26 Qualitative data will be imported to 334 analytic software (Dedoose), where the investigators and coordinator will perform iterative, inductive 335 coding to identify recurrent themes, categories, and relationships among themes and categories. The 336 analysis process will include open coding (identifying major themes and component codes), selective 337 coding (refining themes and codes under each theme), and axial coding (uncovering relationships 338 among themes and codes.) 82, 85 To ensure trustworthiness (a qualitative concept similar to reliability in 339 quantitative analysis [85][86][87][88] ), we will perform a "member check" of the results with participants (n=6) 340 selected for diversity of participant type. We have extensive experience using grounded theory to 341 develop an understanding of palliative care and interventions for improving this care. 71,89-97 342 343 e. Sample size considerations: 344 345 Sample size estimation for pilot trials are often determined by requirements associated with feasibility 346 and acceptability assessments, and one should use caution in powering pilot studies based on key 347 outcomes. 79-81 However, we chose to power this pilot study based on a "process of care" outcome -348 the documentation of goals-of-care discussionsto assess efficacy and facilitate future studies. In 349 our recent outpatient trial, the Jumpstart intervention was associated with a significant increase in 350 documentation of goals-of-care discussions from 17% in the control group to 62% in the intervention 351 group (p<0.001). 2,12 Based on our preliminary data, we estimate that 50% of the control group in this 352 inpatient setting will have documented goals-of-care discussions by the time of death or discharge.

353
This estimate provides for estimates that are maximally conservative, since power increases further 354 from the 50% mark. We powered this trial to determine if the intervention is able to increase this 355 proportion to 75%, with 95% confidence intervals and power of 80%: this would require a sample size 356 of 55 patients in each group with complete data. We plan to recruit 75 patients in each group to 357 ensure complete data on 55 patients in each arm. This sample size of 75 patients in the intervention 358 group will also provide adequate power to assess feasibility and acceptability. 79-81 359 For Aim 3, it is important to achieve theoretical saturation (no new themes emerging). 85,98 We will 360 monitor for saturation, and if saturation is not achieved, we will recruit additional participants. 361 362 f. Data management and quality control to achieve scientific rigor 363 364 This project requires the creation, maintenance, and analysis of a database that includes a variety 365 of measures from multiple sources. This study, like all studies, depends on the quality of the data and 366 therefore systematic data collection, quality control, and data-management procedures will be 367 implemented: 1) protocols for data collection; 2) rigorous training, certification, and periodic re-training 368 of study staff, with ongoing monitoring of adherence to protocols; 3) regular review of questionnaire 369 response rates, respondent burden, 99 and missing items to identify and correct problems; 4) 370 verification of all data through custom-designed data entry systems; and 5) monthly team meetings 371 and reports to provide feedback to study staff to ensure problems are resolved quickly. To ensure 372 reliability and validity of data, we will use our current methods for training and quality control. 100-104 373 Staff conducting EHR review will undergo >80 hours of training: instruction on the protocol, guided 374 practice abstraction, and independent abstraction with reconciliation by a trainer. A 10% random 375 sample will be dual-abstracted. We will blind abstractors to randomization status and survey results. 376 377 g. Protocol modifications 378 379 NLP/ML algorithm: Our NLP/ML algorithm has required ongoing refinement. Hence, for this study we 380 implemented manual abstraction for the purposes of collecting our primary outcome measure, EHR 381 documentation of a goals of care discussion. We are using data gathered via manual abstraction to 382 refine our NLP/ML algorithm. The performance characteristics of this new algorithm are improving 383 over time. A recent version of this algorithm, compared to the standard of manual abstraction, shows 384 a sensitivity of 57%, specificity of 99%, positive predictive value of 53% and negative predictive value 385 of 99%. This produced a positive likelihood ratio of 0.43 and a negative likelihood ratio of 0.71. 386 Although these are good test characteristics, sensitivity and positive predictive values are too low to 387 use this algorithm for outcome adjudication in a randomized trial.

389
Time to event: Our initial intent was to assess time to goals-of-care discussion as a secondary 390 outcome. However, given the low proportion of events, this analysis was not included. 391 392 h. Anticipated limitations 393 394 Sample size: The sample size will limit our ability to detect differences between groups for most 395 outcome measures. However, the goals of this pilot study are to assess feasibility and acceptability of 396 the intervention, evaluate for increased documentation of goals-of-care discussions, and develop 397 insights for how to make the intervention more effective. The sample size is adequate for these goals. 398 Generalizability: This study occurs in a single healthcare system which limits generalizability, but 399 includes two diverse hospitals that use both Cerner and EPIC EHRs, which enhances generalizability. 400 Including additional healthcare systems is not feasible for this pilot. 401 Scalability of surveys: Study staff will distribute surveys which would not be scalable for broad 402 implementation of the intervention in clinical practice. However, Aim 3 will provide insights into how 403 best to address this limitation for the subsequent hybrid effectiveness-implementation trial.

404
Quality of communication: Our NLP/ML and manual abstraction approaches identify goals-of-care 405 discussions without assessing their quality. Since our prior trials demonstrated increased patient-406 assessed quality with the intervention, this is less of a concern. 2,12 Future NLP/ML advances may 407 permit quality assessments. 408 409 i. Anticipated findings 410 411 This proposed pilot study is an innovative intervention to improve goals-of-care discussions for 412 seriously ill hospitalized patients and their families. The intervention uses the EHR to identify patients 413 who should have documentation of a goals-of-care discussion but do not, and then prompts and 414 guides this discussion with a bilateral intervention that provides patient-specific support to clinicians, 415 their patients or their family members. The goal of this pilot study is to create the foundation for an 416 innovative effectiveness-implementation trial that would be submitted to the NIH. 417 418 j. References 419 420