Association of an Advance Care Planning Video and Communication Intervention With Documentation of Advance Care Planning Among Older Adults

Key Points Question Can an advance care planning (ACP) video and communication intervention promote ACP for elderly patients during the ongoing COVID-19 pandemic? Findings This pre-post, open-cohort nonrandomized controlled trial compared ACP documentation during three 6-month periods: pre–COVID-19 (14 107 patients), COVID-19 wave 1 (12 806 patients), and an intervention period (15 106 patients). The ACP documentation rates were 17.9% in the pre–COVID-19 period, 12.5% in the COVID-19 wave 1 period, and 23.7% in the intervention period; ACP rates during the intervention period were highest compared with the 2 other periods. Meaning The use of an ACP video and communication intervention may promote ACP for elderly adults during the evolving COVID-19 pandemic.


Objectives
The overall objective of this application is to reduce the burden of COVID-19 and its consequences for an aging U.S. population that may prefer to forgo aggressive interventions that are unlikely to help, and to die with loved ones present outside of the hospital setting. To accomplish this, we propose to conduct a Pre-Post trial using an open cohort design of a comprehensive telehealth advance care planning (ACP) Program among older patients in New York, one of the nation's COVID-19 epicenters. To address this gap, we have developed a Comprehensive Telehealth ACP Program for COVID-19 that implements ACP routinely into primary medical care. The Comprehensive ACP Program combines two widely disseminated interventions to assess their impact when used concurrently. One is VitalTalk clinician communication skills training (www.vitaltalk.org) during which the primary care clinicians will practice ACP with simulated patient actors under the guidance of a trained VitalTalk facilitator. The second is the ACP Decisions video decision aids (www.acpdecisions.org) which are directed to patients and provide education about ACP. This inclusive ACP approach treats patients and clinicians as equal stakeholders providing both with the communication skills and tools needed to make decisions about COVID-19 medical care before the toughest choices arise.
The training is four cumulative hours, most of which is spent in communication skills work, and the remainder learning how to introduce the videos into one's practice. No patients participate in the training. The ambulatory practices included in this intervention gain access to the ACP Decisions videos, which can be introduced to patients. Videos may be seen in clinic or sent to patients' homes via an electronic or paper code. We have shown the efficacy of the elements of the Comprehensive ACP intervention in several randomized controlled trials (RCTs) for older patients. (1)(2)(3)(4)(5) We will train 150 primary care clinicians caring for 13,000 diverse patients over the age of 65 from Northwell Health, a health care system in which we are already embedded with our ongoing grant. We will use Natural Language Processing (NLP) to abstract patient outcomes from the Electronic Health Record (EHR). We hypothesize that a telehealth ACP Program will improve and sustain rates of ACP from the time that the intervention is implemented (11/ Exploratory Aim: Qualitative assessment of barriers and facilitators for intervention success. Our goal is to identify and describe clinician implementation experiences .
IMPACT: COVID-19 poses a unique dilemma for older Americans, who must balance their desire to live against the risk of a lonely and traumatic hospital death. Clinician communication training and video decision support is a practical, evidence-based, and innovative approach to assist patients facing this most difficult of all choices. Major strengths of this proposal are: the highly experienced team making this project feasible; the present infrastructure already embedded at Northwell Health; and, the potential immediate deployment of the intervention, if successful, across the country. This work holds the promise of improving the quality of care provided to millions of Americans during the COVID-19 pandemic.

Background SIGNIFICANCE
Epidemiology: COVID-19 disproportionately affects the elderly and those with serious illnesses such as heart disease, cancer, and dementia.(6) These diseases are common, morbid, and costly conditions, especially in people over the age of 65.(7) When COVID-19 strikes, older patients die at higher rates. (8) The surge in patients with COVID-19 poses a significant challenge to clinicians and health care institutions.
Lack of effective ACP during COVID-19 has led to burdensome, costly, and often avoidable interventions: Medical technology enables an extraordinary array of possible interventions for older people. However, particularly in the setting of COVID-19, the burdens of treatment may outweigh its beneficial effects. In the elderly, even the most aggressive treatment may not prevent death from the clinical complications of COVID-19, which include respiratory failure, renal failure, and cardiac arrest.(56) Furthermore, and unique to this pandemic, hospitalized patients may not receive visitors, thus introducing loneliness and a sense of abandonment to what is already a traumatic death. Given this risk-benefit profile, many high-risk patients, particularly among the elderly, may choose to decline aggressive treatments or even hospitalization. Yet, without the presence of a documented ACP discussion, the default approach is to perform all invasive procedures. Research suggests that "rescue" interventions such as mechanical ventilation and attempted cardio-pulmonary resuscitation (CPR) are inappropriately used even when not desired. (57)(58)(59) Despite the mounting evidence suggesting that older patients with serious illness prefer palliation, they often receive burdensome interventions and suboptimal control of pain and other distressing symptoms, particularly in the absence of high-quality ACP. (57,(60)(61)(62) The converse is true for patients with documented end-oflife decisions. (11,12,21,(63)(64)(65)(66) ACP is necessary to avoid unwanted interventions; high-quality ACP increases patient safety by ensuring that patients receive effective medical care that matches their goals.
Clinician communication training and video decision support improves ACP: Traditional approaches to ACP, which often rely on ad hoc verbal descriptions of hypothetical clinical situations and treatment choices, is limited because complex scenarios are difficult to envision, provider information is inconsistent, and verbal explanations are hampered by literacy, emotional, and language barriers.(32, 67-68) Patients often do not fully understand the choices presented in advance directive legal documents, and the quality of communication about these choices is suboptimal. (69) Over the past few years, investigators have recognized the shortcomings of prior efforts and developed new interventions to better facilitate ACP. (70) The Comprehensive Telehealth ACP Program for COVID-19 proposed for this study trains patients and clinicians to communicate about ACP in the context of COVID-19. Prior work, including several RCTs, supports the efficacy of the ACP Decisions videos and the VitalTalk program in older patients. (71)(72)(73)(74)(75)(76)(77)(78)(79)(80)(81)(82)(83)(84)(85)(86)(87) This application represents the first Comprehensive Telehealth ACP Program for COVID-19 that combines two well-tested, evidence-based, and complementary interventions using an online platform.
Our research group's expertise in Natural Language Processing (NLP) and ongoing U grant at Northwell Health enables a study of this scale and on a shortened timeline: For the proposed trial, we will use NLP, a form of computer-assisted abstraction, to detect our primary outcome, ACP documentation in the EHR. Our research group has published the seminal studies using NLP in ACP, and we have already applied this methodology at Northwell Health in our ongoing study (1UH3AG060626-02). Northwell Health, the largest health care system in New York, is an ideal setting for this trial. The ability to use NLP to quickly and rapidly abstract outcomes from a large number of EHR records, and our present work with Northwell Health, lays the groundwork for this trial.
A Pre-Post, open-cohort, pragmatic trial is well-suited to evaluate an ACP Program intervention in older patients during the COVID-19 pandemic: Several notable advantages of this design exist for testing communication interventions in older patients during a pandemic. (88)(89)(90)(91) In response to COVID-19 and the increased mortality, we expect that Northwell Health ACP activity increased naturally. Two six-month control periods prior to the wide-scale implementation of the intervention, allow us to measure a baseline rate of ACP activity and identify the "pre-COVID-19 ACP baseline rate" and a "COVID-19 ACP baseline rate." We chose September 15, 2019 as the start of the pre-COVID-19 control period since it is about one year before the intervention start, and we chose March 15, 2020 as the start of the baseline control period since the earliest ACP activities due to COVID-19 began during that month. (92,93) We will then compare the ACP rate during the implementation period, which begins November 1, 2020 and continues for six months, to the rates in the previous two control periods. Given that the intervention is entirely online, no logistic barriers prohibit us from training providers on a shortened timeline.(94) Clinicians will be trained during the early weeks of the trial, avoiding any contamination during the control period. In addition, the low-risk intervention being tested will be implemented at the clinic level and our ongoing study received IRB-approval exempting it from individual consent procedures when using an open-cohort design. Thus, the proposed trial is an optimal design for a COVID-19 study and a logical next step toward understanding the real-world application of the Comprehensive ACP Program. Taken together, this work has the potential to improve the quality of care provided to millions of older Americans during the COVID-19 pandemic.

Inclusion and Exclusion Criteria
Clinic Eligibility: Clinics affiliated with Northwell Health may be included in the study.
Clinician, Staff Eligibility: Any staff member identified by the site-PIs (Drs. LaVine, Carney and Burns) who are affiliated with Northwell Health.

Patient Eligibility:
To be eligible for this study, individuals must be aged 65 or over, affiliated with the Northwell Health clinic. For each of the three periods of the study, those patients over the age of 65 who are engaged (e.g., seen in person, telehealth, etc.) with the clinic during that time period will be included in that time period.
• We will not be including individuals who are not yet adults (infants, children, teenagers) • We will not be including pregnant women • We will not be including prisoners

Study-Wide Number of Subjects*
We will study data from 13,000 patients aged 65 or older for our primary (completed advance care plans) outcome. ACP can include designating a health care proxy, discussion of advance directives, goals-of-care discussions, CPR discussions, palliative care discussions and referrals, as well as hospice. Each of these ACP outcomes will also be compared in secondary analyses using NLP. These data will be obtained from the EHR. All patient participants will only be included for medical record review.
In addition, we will recruit up to 30 clinicians, from participating clinics, who completed the intervention training. We will ask them to complete a qualitative interview assessing their experiences with the ACP intervention.

Study-Wide Recruitment Methods*
Patients will not be recruited into the study, rather we will seek a waiver of individual informed consent and HIPAA authorization since the research involves no more than minimal risk to subjects.
The 30 clinicians recruited for the qualitative interviews will participate only after providing individual verbal informed consent.
We do not foresee a significant challenge with ensuring follow-up as all patient data is collected passively via the EHR, and the clinician interviews are single episode data collections.
We also have a "Communications Log" where we track the important calls, meetings and interactions.

Participating Institution
Each Participating Institution is expected to comply with all applicable federal regulations and DF/HCC requirements, the protocol and HIPAA requirements.
The general responsibilities for each Participating Institution may include but are not limited to: • Document the delegation of research specific activities to study personnel. Have office space, office equipment, and internet access that meet HIPAA standards. 5. Participate in any quality assurance activities and meet with monitors or auditors at the conclusion of a visit to review findings. 6. Promptly provide follow-up and/or corrective action plans for any monitoring queries or audit findings.

Long-term storage of Data:
Data stored on the DFCI server will reside there only for the periods they are required to be there for study usage. Data will be securely removed from these servers on a per-item basis. Removed data will be securely transferred to DFCI long-term servers for storage.

Dana Farber, Northwell Health, Boston Medical Center (BMC) and Massachusetts General Hospital (MGH) Roles:
As in the ACP PEACE Trial, Dana Farber is the IRB of record and Northwell Health has ceded IRB review to DFCI. Northwell Health will review clinical notes from the medical records for ACP document using the NLP software ClinicalRegex. The results of this NLP review will be stripped of any PHI sent to DFCI for analysis. The review of these records will be used to study the impact of the clinician trainings and patient educational videos currently being implemented at Northwell Health primary care practices on patient advance care planning outcomes. Study staff at BMC will assess the implementation of the intervention across hospital's outpatient internal medicine clinics. BMC will receive and analyze limited data set(s). All data that BMC receives will be de-identified and there will be no PHI. At MGH, Dr. Volandes is co-leading the entire study along with Dr. Tulsky. He will oversee the implementation of the videos with NWH. No identified data or PHI will be involved in MGH's role.
The clinician interview recordings will be captured via Northwell's Teams platform. The recordings will be transcribed and Northwell will provide access to a shared drive. The shared drive will contain the de-identified interview transcriptions. There will be no transfer of data, the data will stay at Northwell.
The audio recordings will be saved in a secure HIPPA ePHI drive (OneDrive) and will be assigned a unique identifier. A professional research transcriber will then generate transcripts based on the audio recordings. The transcripts will be assigned unique identifiers and be uploaded into a secure HIPPA ePHI drive for analysis by the BMU study team. Only IRB approved study personnel will have access to the study data.
The audio recordings will be transcripted and then the recordings will be destroyed to be de-identified. We will redact the names on the transcriptions.

Study Timelines
The study has three distinct time periods During the first month of the project, we will organize the logistics for study implementation and prepare NLP and other tools. By month 2 we will have organized clinician trainings-clinician re-training will then reoccur between months 3-8. Months 3-10 will include data collection, cleaning, and analysis. Manuscript preparation and dissemination of the trial results will occur during the final two months.

Study Endpoints Primary and Secondary Endpoints:
We will conduct a Pre-Post, open-cohort, pragmatic trial of the Comprehensive Telehealth ACP Program to promote ACP documentation in 13,000 patients aged 65 and over affiliated with 150 outpatient primary care clinicians from Northwell Health. The primary outcome for this pragmatic trial is documentation of ACP in older patients. We will assess ACP documentation for the two periods prior to the intervention (September 15, 2019 -March 14, 2020, pre-COVID-19 baseline rate; and, March 15, 2020 -September 14, 2020, COVID-19 rate), and compare them to the six-months after the implementation of the intervention (November 1, 2020 -April 30, 2021). Our primary outcome is documentation of ACP activity as determined through NLP of EHR records confirmed by human coders. Discussions of health care proxy, advance directives, goals of care, completion of advance plans, and changes of resuscitation orders, palliative care and hospice discussions, will all count in this composite measure. These outcomes will be ascertained from the records of 13,000 patients across Northwell Health's clinics.

Secondary Safety Endpoints:
There is a risk of patients becoming uncomfortable with discussing these topics; however, ACP is routine and part of the standard of care.

Procedures Involved
The present intervention has already been IRB-approved and being successfully conducted at Northwell Health in the system's cancer clinics (1UH3AG060626-02). The present proposal expands our ongoing work from the cancer setting to the outpatient primary care setting.
A Pre-Post, open-cohort, pragmatic trial study design will compare the rate of ACP documentation during the intervention to two control periods. Two six-month control periods prior to the wide-scale implementation of the intervention, allow us to measure a baseline rate of ACP activity and identify the "pre-COVID-19 ACP baseline rate" and a "COVID-19 ACP baseline rate." We chose September 15, 2019 as the start of the pre-COVID-19 control period since it is about one year before the intervention start, and we chose March 15, 2020 as the start of the baseline control period since the earliest ACP activities due to COVID-19 began during that month. We will then compare the ACP rate during the implementation period, which begins November 1, 2020 and continues for six months, to the rates in the previous two control periods.

Data Collection Protocol:
Patients: Data will be collected from 13 months prior to the implementation start date (i.e., September 15, 2019) and continue for six months after the implementation start date (i.e., November 1, 2020 -April 30, 2021). Our primary (ACP documentation) outcome will be abstracted from Northwell's data platform, which has real-time data for these variables, using NLP. We are already using the process for our ongoing trial.

Qualitative Interviews:
Northwell will identify the clinicians from those who completed VitalTalk training, email the Detailed Information Sheets and facilitate scheduling the interviews. The investigator conducting the interview will review the information on the Detailed Information Sheet with the clinician on the same day as their interview.
VitalTalk communication skills training: Clinicians will receive VitalTalk intensive communication skills training via highly structured Zoom conference. They will learn skills relevant to discussions about ACP and COVID-19 including delivering serious news, eliciting goals of care, and managing difficult conversations via telehealth platforms. The training is based upon the pedagogical framework that skill acquisition requires observation, practice, and feedback. The synchronous training will be done in two two-hour or one four-hour, small group sessions where clinicians practice specific skills such as "ask-tell-ask" or responding to patient emotion and learn to follow "talking maps" that provide a guide through the conversation about goals of care. In these online sessions with simulated patient actors, the clinicians will set a learning goal, practice the skill in question, receive constructive feedback in a highly structured format, and finally have opportunities to "rewind and replay," so that they can experience success and see its effect on the patient. Participating clinicians will also receive the VitalTalk Tips smartphone app that provides daily skill reminders and access to all of the COVIDspecific communication skills and will attend a one-hour refresher training webinar every month to address challenges that may have arisen and to sharpen their newly learned communication skills. This online course will teach content equivalent to the current VitalTalk Mastering Tough Communication courses that are used in multiple health care settings.

Suite of ACP videos for COVID-19:
All clinicians will also be trained remotely on the ACP Decisions Video Program using the ACP app. Training will instruct clinicians on how to: 1. Introduce the COVID-19 videos to patients and caregivers; 2. Select the appropriate video(s) from the entire suite according to patients' needs; and, 3. Prescribe videos for patients and caregivers using the electronic platform. The suite of ACP videos is designed to address common ACP decisions confronting patients at risk or with COVID-19 and their caregivers. The videos are intended to be an adjunct to clinician counseling, not to replace it. The videos range from 4-6 minutes and were developed using an iterative process of drafting and review by infectious disease experts, geriatricians, critical care doctors, palliative care clinicians, patients, and family members. The content in the videos is intended to be objective and balanced and is scripted at a sixth-grade level of health literacy or less. The videos are available in 25 languages. The videos have closed captioning, and can be viewed in real time at an office visit with an iPad, or if done remotely (e.g., by phone or telemedicine) via a web-link. Northwell Health uses a vast network of telemedicine tools to communicate with patients including smartphones and tablet computers, which all support video viewing.
Intervention training: Remote clinician training will occur during the early weeks of the trial. Clinicians will be trained together in small groups. Training will instruct the clinicians on how to: i. more effectively communicate with patients regarding COVID-19, ii. have ACP conversations with patients, iii. introduce the videos to patients and families, iv. use the videos as an adjunct to ACP counseling by clinicians, v. select the appropriate video(s) from the entire suite as according to patients' needs, and, vi. use the app or electronic platform for prescribing videos to be seen at home (telehealth visits), or to be viewed in clinic with an iPad. The Northwell Health Telehealth program is a mature program in which the majority of patients have access to either a smartphone or tablet for virtual clinical care. The intervention training program will leverage the infrastructures and tools already developed in our ongoing research study within Northwell. Each month, one-hour refresher trainings will be available to all clinicians. We also expect the clinics to have some staff turnover and new hires over the course of the trial; thus onboarding and training will be made available on an ongoing basis.

Performance of NLP
Dr. Lindvall's lab at DFCI has published several studies on the use of NLP to capture documentation of ACP in clinical notes. (91 -93) Compared to standard medical record review her team has achieved a sensitivity 97% -99% and specificity 85% -100% at the note level. At the patient level, both sensitivity and specificity are close to 100%. Patients accrue hundreds to thousands of notes which makes standard medical review not feasible, and this method dramatically increases efficiency and sensitivity.
Advances in computer science techniques such as NLP allow for efficient and accurate analysis of the free-text from clinical notes in order to detect our primary outcomes. The increasing use of EHR combined with computational advances in NLP can accurately quantify our primary outcome.
The below procedures will be completed at Northwell Health by the respective site-Investigators. No patient identifiable information will leave the HIPPA-secure firewall of the health care system.
Each site-Investigator will use NLP to identify the following primary outcome.
Data will be abstracted from the EHR data repository. Demographic information and baseline characteristics relevant will be collected. Analysis of free text regarding ACP will be extrapolated using computational methods.
Individual variables have been selected as they are basic demographic and baseline characteristics of study participants.
The Investigators will use computational methods (NLP) to identify our outcomes. Once these variables have been abstracted from the health care system's EHR, the data will be collected for each patient and the patient will be given a random unique identifier. A cross-walk of the unique identifier and the identifiable information of the patient will be securely stored at the health care system and not transferred to DFCI.
The deidentified data for each patient will then be sent to DFCI and MGH for further analysis.
Only researchers listed on this IRB will be able to access the data. The risks will be minimal as the data will be stored and analyzed on the HIPAA secure cluster at each health care system. None of the data will be stored in paper form. The data and identifiers will be kept for three years on the HIPPA secure cluster computer After the three years, the data will be permanently destroyed.

DATA ANALYSES
Aim 1: To test the combined effects of an ACP Program on rates of ascertainable quality measures of ACP among 13,000 patients at risk or with COVID-19. We will use an intention to treat approach. All patients over 65 engaged with the clinic over the course of each time period in each of the three study periods will be included in the analysis. We will use generalized linear mixed models to compare outcomes between intervention and the two control periods. The basic model is depicted in Equation (1) ( where denotes the response from individual k from cluster (PCP) j at period i. The model includes a fixed effect α for period i, which indicates the difference between periods. Our primary comparison is between the post-intervention period and the two control periods. The model also includes a random effect β for cluster j to take into account the patients within PCP clustering. It is likely a large proportion of patients will overlap in the adjacent study periods; therefore, the model includes a random subject effect γ for individual k to take into account the repeated measures from the same subject. We will limit the analysis to those who appear in two or more comparison periods as a secondary analysis. The model also includes an additional term of Χ which represents the patient characteristics at period i to adjust for potential confounding factors. We will use a logit link ( ) for the binary outcomes which include our primary outcome of ACP documentation.

Sample Size Calculation & Power Analysis:
We estimated approximately 7,800 patients from 150 PCPs are eligible for the study at each period and 85% patients overlap from one period to the next. With 3 time periods, we anticipate a total of 10,139 unique patients will be included in the study. For our primary comparison between COVID-19 period and post intervention period, we expect 7,800 patients in each of the two periods with 6,630 appearing in both periods. In the most conservative scenario with each PCP contributing an average 44 patients, the design effect is 3.2 assuming an intra-cluster correlation of 0.05, which corresponds to an effective sample size of 2,098 appearing in both periods. Preliminary estimates indicate the rate for ACP documentation (primary outcome) is around 10% pre-COVID and 20% during COVID. The study will have more than 95% power to detect a 5% absolute increase in outcome with a two-sided significance level of 0.05. The power will be even higher for a 10% post intervention increase, which is our threshold of a clinically meaningful difference.
Exploratory Aim: To examine facilitators and barriers to intervention success, we will conduct a qualitative analysis of interview transcripts collected from 30 participating clinicians. An examination of their perceptions of the training and video intervention, as well as facilitators and barriers to implementation will help to contextualize, and possibly explain, the trial findings.

Aim 2:
To characterize ACP documentation rates among under-represented minority groups. The analysis will use a difference in differences approach comparing the changes from the control period to the intervention period between White and non-White racial minority and between non-Medicaid and Medicaid patients. This will translate into adding a subgroup fixed effect to (Eq 1) and test the interaction between period and subgroups. We will explore the intervention effect in each racial minority (Black, Hispanic, Asian) separately.

Sample Size Calculation & Power Analysis:
Preliminary estimates indicate 30% of the population is non-white racial minority and approximately 30% of the population were on Medicaid. Assuming the rate for ACP documentation is 21.5% among Whites and 16.5% among non-White (overall 20%) during COVID, and both groups increase to 30% postintervention, our simulation study shows that we will have >90% power to detect a 5% difference in differences after taking into account of clustering. We anticipate a larger difference in differences between Medicaid and non-Medicaid patients; therefore, the power for that analysis will be even higher.

SECURING THE DATA EHR Data Transfer Procedures
1. Representatives from Northwell Health will make six-month data "dumps", using appropriately secure tools, of EHR data onto selected servers. 2. Scheduled job run by DFCI will initiate the ingestion of new data as made available. 3. DFCI's system manager replaces person identifiers, including all HIPAA face identifiers, with an alphanumeric ID.

Withdrawal of Subjects "Broadcast notifications":
We will be providing any potential patients (participants) with the opportunity to opt out of this project entirely (i.e., to have their deidentified data from the EHR not included in our main analysis). We have created "broadcast notifications" in the form of a poster in the clinic/patient areas giving brief commentary about the trial and their choice to participate or not to participate in having their data included in our analysis.

Potential Risks
The potential risks are minimal given the fact that the intervention promotes learning about ACP, improves communication for patients and their families regarding ACP and self-determination, and the concordance between patient's wishes and the care they receive.
The major potential risk for subjects is loss of confidentiality. Loss of confidentiality is very unlikely because specific procedures have been implemented by the research team to prevent such disclosure and these measures will be maintained during the proposed study.
Another risk is being upset by the questions, which are part of the standard of care. Probability of occurrence is minimal. We have conducted a series of clinical trials for patients with advance illnesses and have rarely had patients get upset due to the topic. In each of these cases, the participant was interested in continuing after a short break.
Clinicians may find some of the interview content difficult as we will be discussing their experience of advance care planning with patients, some of whom may have been very sick or died from COVID. During the interview we will stop at their request or skip questions that make them uncomfortable. If needed, we can recommend that they contact their employee wellness or Mental Health program.

Protects Against Risk:
We will emphasize that participation in this research is voluntary. Furthermore, Northwell Health allows patients to decline having their deidentified data including for research purposes.
This study includes data from hospital data systems. We will access patient records for research study data within the explicit guidelines of the limited data specified.
Risks to confidentiality of the data collected throughout the proposed study will be addressed as follows: all information in the database will be indexed by subject identifier only, so that even if the database server is compromised subjects cannot be identified, thus maintaining the privacy of their information. Also, assurance and confidentiality of information will be made to all subjects. Data will be handled with the same confidentiality accorded to patient's medical records. Specific procedures protecting subject confidentiality will be as follows: • Access to data files will be secured with a password-filing system (that logs entry) and is restricted to authorized staff only. • Necessary hard-copy records containing study data of any type will be kept in locked files. • Master lists linking subject information with ID number will be numbered consecutively and prepared before data collection (to ensure accurate accounting). These lists will be kept locked, in duplicate, with access only by the PIs and the other investigators. • All project staff will sign an oath of confidentiality to ensure their understanding of the terms of confidentiality required. They will be trained in specific procedures to ensure confidentiality. • Sign-out procedures for all access to data files will be strictly enforced.
• All reports and publications will preserve the subjects' anonymity.
Any breach of confidentiality will be subject to a root cause analysis and preventive measures taken as appropriate. All of these procedures are likely to be effective based on our prior research experience.

Potential Benefits to Subjects Potential Benefits of the Proposed Research to Human Subjects and Others
There is the potential for patients and clinicians in the clinics to benefit from the study by having their treatments better aligned with their preferences. Society, medical science and the health care system may benefit from the information obtained in the research on

Importance of the Knowledge to be Gained
There is the potential to validate an intervention that could ensure that treatments are better aligned with patients' preferences. The minor risks for the participants in this study may be considered counterbalanced by the potential direct benefits and knowledge gained. The results gleaned from the study are intended to improve the ACP of the overall outpatient clinic population. Thus, the risk/benefit balance for this study appears favorable.

Sharing of Results with Subjects
The results will be shared via a patient friendly pamphlet that explains the results of the study and is disseminated in participating clinics. An e-version will also be available on any websites affiliated with the practice.

Location:
Patients from the Northwell Health clinics will be enrolled.

Eligibility Estimates:
We anticipated that a total of 150 clinicians to be included in the trial.

Qualifications
James A. Tulsky, MD, Co-Principal Investigator: Dr. Tulsky is Professor of Medicine at Harvard Medical School and Chair, Department of Psychosocial Oncology and Palliative Care at DFCI. He has led multiple federally funded studies examining patientprovider communication, and is a Founding Director of VitalTalk, a non-profit organization devoted to teaching physicians communication skills (www.VitalTalk.org). Dr. Tulsky receives no compensation from VitalTalk. Dr. Tulsky will provide scientific leadership over all aspects of the study. He will oversee the conduct of the study including protocol development, training of research staff, codebook development, data analysis and manuscript writing. Angelo Volandes, MD, MPH, Co-Principal Investigator. Dr. Volandes is a general internist and clinical researcher with past training in medicine, decision-making, bioethics, and film-making. His research career has been dedicated to developing and evaluating video decision support tools for life-limiting conditions. He currently leads an innovative team of video decision scientists at the Massachusetts General Hospital. He has successfully led many studies funded by the NIH evaluating the efficacy of these tools, the results of which have been published in top-tier journals.

Feasibility:
• The present methodology has already been IRB-approved and being successfully conducted at Northwell Health in the system's cancer clinics (1UH3AG060626-02). The present proposal expands our ongoing work from the cancer setting to the outpatient primary care setting. Thus, the timeline of the project is quite feasible and meets the requirements of the one-year grant. • Our research group's expertise in NLP and ongoing U grant at Northwell Health enables a study of this scale and on a shortened timeline: For the proposed trial, we will use NLP, a form of computer-assisted abstraction, to detect our primary outcome, ACP documentation in the EHR. Our research group has published the seminal studies using NLP in ACP, and we have already applied this methodology at Northwell Health in our ongoing study (1UH3AG060626-02). The ability to use NLP to quickly and rapidly abstract outcomes from a large number of EHR records, and our present work with Northwell Health, lays the groundwork for this trial.

Recruitment Methods
All subjects engaged with the clinic will be recruited. No particular recruitment methods will be used.

Local Number of Subjects
There will be no local subjects recruited for this project.

Provisions to Protect the Privacy Interests of Subjects
The risks in this study are minimal and non-medical in nature. The primary risk is loss of confidentiality. To minimize the likelihood of a breach, we will collect only electronic data and anonymous data when possible. Only the minimum amount of PHI necessary will be collected from study participants, including clinicians and patients, and all data will be transmitted via secure, institutionally approved methods. We will emphasize that participation in this research is voluntary.
Qualitative interviews with the 30 clinicians will be conducted by personnel who have no affiliation to the clinicians' employer. Individual data from these interviews will not be shared with Northwell Health management. The transcripts will be deidentified, analyzed in composite, and only seen when published in this form in the medical literature.
We will transmit no identifiable data back to the clinic sites to avoid any risk of retribution, retaliation, or adverse consequence to disclosures which may come about through this process. Research staff, investigators, data management staff, and those with responsibility for analyses will have access to subject identifiers and private health information.
Oversight and Protocol Compliance: All research personnel will participate in human subjects training, an annual booster training, one-time data integrity and security training, and training on the protocol, which will include simulation and practice for all protocol procedures including obtaining informed consent and confirming comprehension. We will have regular team meetings and regular surveying reliability checks to ensure protocol compliance. The procedures described here address our efforts to minimize the risk of breach of confidentiality and to ensure that procedures remain in place for the protection of human subjects. All data management will be conducted at DFCI. The Center's researchers and staff have many years of experience working with similar data files.
Adverse Event Reporting: This study presents minimal risk to participants, and there are no adverse events given that ACP is part of the standard of care. Nonetheless, the PIs will monitor and report unforeseen adverse events to the DFCI IRB. If there are any concerns about privacy, will be refer the patient to the DFCI Privacy Officer, and we will refer any significant complaints to the DFCI Office of Research Studies.

Consent
(1) Patients (via EHR): There are special informed consent considerations in this pragmatic Pre-Post trial: the intervention is of low risk and will be implemented as the standard of care for the whole clinical unit, and data for our primary outcome and related outcomes derived from the EHR is ascertained from existing sources. Thus for this aspect of our proposal, we will seek a waiver of individual informed consent and HIPAA authorization after careful review of the criteria to do so. The research involves no more than minimal risk to the subjects as described above. We do not believe the waiver will adversely affect the rights and welfare of the subjects. As a pragmatic trial of thousands of patients, this research could not practicably be carried out without the waiver nor without access to and use of PHI of patients. Finally, we have developed a plan to protect identifiers from improper use and disclosure. It is our assessment that this study meets the regulatory requirements for the preparatory for research provisions of the HIPAA privacy rule and will work with the institutional IRB and Privacy Officer to establish that this does meet the standards for the security of protected health information. In accordance with this provision, only researchers and staff who are a part of the "covered entity" will use the preparatory research provision to contact prospective research subjects and no identifiable PHI will leave any covered entity without consent and an approved procedure. We are currently conducting a study with similar EHR data abstraction via NLP and have been granted a waiver of consent and a HIPAA waiver for these activities.
(2) Clinicians (Qualitative Interviews): We will obtain individual verbal informed consent from all clinicians research subjects. While clinician subjects will be presented with all required elements of consent via a Detailed Information Sheet that is (emailed, mailed, described over the phone, etc), a signed document will not be obtained.
Non-English Speaking Subjects = N/A Subjects who are not yet adults (infants, children, teenagers) = N/A Cognitively Impaired Adults = N/A Adults Unable to Consent = N/A

1.DATA OBTAINED FROM THE EHR:
To obtain personal health information from the EHRs, a waiver of HIPAA authorization will be sought.

DATA OBTAINED FROM CLINICIANS THROUGH QUALITATIVE INTERVIEW:
For the clinician participants (N=30) being interviewed to answer the exploratory aim about implementation, verbal informed consent will be obtained.