A Cluster Randomized Trial to Assess the Impact of Facilitated Implementation on Antibiotic Stewardship in Wisconsin Nursing Homes Primary Investigator :

Key Points Question Does external facilitation improve adoption and effects of a complex antibiotic stewardship intervention in nursing homes? Findings This trial protocol describes a cluster-randomized hybrid type 2 effectiveness-implementation clinical trial of implementation of a multicomponent toolkit focused on improving the recognition and management of suspected urinary tract infection (UTI) in nursing homes. The trial seeks to evaluate whether delivery of external facilitation—coaching, collaborative learning, and peer comparison feedback—to implement this toolkit results in higher rates of toolkit adoption and reduced rates of urine testing and initiation of antibiotics for treatment of suspected UTI. Meaning If successful, external facilitation could become an effective approach for improving spread and adoption of antibiotic stewardship interventions, as well as other quality improvement initiatives, in the nursing home setting.


I. Background
Antibiotic resistance is one of the three greatest threats to human health according to the World Health Organization [1]. Antibiotic-resistant bacteria and Clostridium difficile are responsible for 2.3 million infections and 37,000 deaths in the U.S. every year [2]. Antibiotics are one of the most commonly prescribed medications in nursing homes (NHs). Up to 40% of NH residents are colonized with antibioticresistant bacteria [3], and NHs play a critical role in the emergence and spread of antibiotic resistance and C. difficile in the communities they serve [4][5][6]. Antibiotics are prescribed to 65% of individuals who reside in a NH for 6 months [7]. Treatment of suspected urinary tract infection (UTI) accounts for nearly half of the antibiotic use in Wisconsin NHs [1]. However, there is compelling evidence that antibiotic prescribing patterns in NHs is sub-optimal (5). Less than half of the antibiotics prescribed for UTI in Wisconsin NHs are justified [6], and improving the management of UTI in NHs has been identified as a major priority by experts in the field [7][8][9].
We along with our community partners have developed a toolkit of evidence-based practices for improving the management of UTIs in Wisconsin NHs (hereafter referred to as the Wisconsin UTI Improvement Toolkit). While use of the practices incorporated in the Wisconsin UTI Improvement Toolkit has resulted in notable improvements in the quality of antibiotic prescribing for the treatment of UTI in a number of studies [10][11][12], more limited effects have been observed in other studies despite employing similar improvement strategies [13][14][15]. Heterogeneity of effects across different settings is a problem commonly encountered with healthcare behavioral interventions and may be due to characteristics of the intervention, the setting into which it is being introduced, and the method of implementing the intervention in that setting [16]. There are a number of challenges to improving quality in NHs, including lack of improvement expertise [17] and high levels of staff turnover [18] to name a few. It is unrealistic to expect that these structural barriers will abate anytime soon. Consequently, it is critically important to identify effective strategies to overcome them if meaningful improvements in the quality of antibiotic prescribing in NHs are to be achieved.

II. Study Objectives
Our long-term objective is to improve the quality and safety of antibiotic prescribing in Wisconsin NHs.
The objective of the current project, which is a critical step in the pursuit of our long-term objective, is to identify effective strategies for supporting the implementation of antibiotic stewardship interventions in NHs. We hypothesize that externally-facilitated implementation based on coaching and peer-to-peer learning will be associated with: (hypothesis 1) better adoption of the Wisconsin UTI Improvement Toolkit and (hypothesis 2) reduced rates of inappropriate antibiotic use as compared to a usual implementation approach that relies predominantly on internal improvement resources available in NHs. We base these hypotheses on studies demonstrating more effective uptake of evidence-based practices with coaching [19][20][21][22] and peer-to-peer learning [23][24][25] in other healthcare settings, particularly resource-limited settings. A preliminary analysis of qualitative data collected from another of the PI's research projects (Agency for Healthcare Research and Quality 1R18 HS022465-01) indicates that clinical coaching is helping champions in study NHs identify barriers to change and develop more effective strategies for overcoming these barriers.
The primary study objectives are to assess the overall impact of the UTI Improvement Toolkit on antibiotic utilization and determine the extent to which effects vary by the implementation approach employed. We will achieve these two objectives through the following specific aims: Specific Aim 1: Evaluate the impact of the Wisconsin UTI Improvement Toolkit on antibiotic prescribing for UTI in Wisconsin NHs.

A. Overall Study Design and plan
We will introduce the Wisconsin UTI Improvement Toolkit (see Section III.B. Wisconsin UTI Improvement Toolkit) in 20 Wisconsin NHs and assess its impact on antibiotic utilization using segmented regression. Two different implementation strategies (see Section III.C. Study Arms) will be employed during the introduction of the Wisconsin UTI Improvement Toolkit in study NHs using a hybrid type 2 effectivenessimplementation study design in which half of the NHs will be randomized to usual (internally-driven) implementation with the other half randomized to externally-facilitated implementation based on coaching and peer-to-peer learning. The effects of the two implementation approaches will be assessed using a mixed methods approach.

B. Wisconsin UTI Improvement Toolkit
The PI (Crnich) has worked with academic, community and public partners in Wisconsin to develop a multicomponent toolkit to improve the management of UTIs in NHs. The WI Healthcare Associated Infections (HAI) in Long term care (LTC) UTI toolkit was developed through a rigorous review of published theory and evidence [7] with input from frontline clinicians who practice in Wisconsin NHs and is structured around a UTI management framework developed by the PI [31]. The toolkit targets three major nursing staff and provider behaviors: (a) standardization of the assessment of the resident experiencing a change-incondition; (b) implementation of delayed testing and treatment of residents experiencing a low-risk changein-condition; and (c) appropriate selection of antibiotics (drug, dose, duration) in those residents where the likelihood of UTI is high. Education, communication, and decision-support tools have been developed in order to achieve improvements in each of these behavioral domains, and the toolkit also includes recommended practices to help prevent UTIs as well as process and outcome tracking tools designed to help staff in NHs assess implementation progress. Further details about the UTI toolkit are provided in the Appendix.

C.1. Control Arm: Internally-driven implementation
Study NHs randomized to the control arm will participate in kickoff meetings introducing the different UTI toolkit components. The meeting will also cover topics on how to assemble a change team, how to use the different tools in the UTI toolkit, and how to track process and outcome measures to monitor progress. NHs assigned to the control arm will have access to a web-based data tracking tool that will allow them to trend a number of study outcomes (e.g., number of monthly urine cultures and UTI treatment events) in their facility over time. Instructions and educational tools will be made available as an online resource but otherwise NHs in the control arm will be expected to implement the Wisconsin UTI Improvement Toolkit using existing internal resources. NHs randomized to the externally-facilitated arm will attend the same kickoff meetings and have access to the same online resources as the control arm but will also be assigned a clinical coach and will have the opportunity to participate in a series of hosted peer-to-peer learning opportunities. The clinical coach will meet regularly with NH staff to guide the facility through implementation of the Wisconsin UTI Improvement Toolkit. Coach supports will occur during a site visit and individual coaching calls.

C.2.1. Site Visit
At the site visit, the coach will review the current UTI diagnosis and management-related workflows and work with SNF staff to identify opportunities to integrate different aspects of the Wisconsin UTI Improvement Toolkit into these existing workflows.

C.2.2. Individual Coaching Calls
Following the site visits, the coach will conduct bi-weekly phone calls with staff in study NHs. These calls will focus on a number of topics, including: a) Establishing a change team; b) Identification of barriers and facilitators of change; c) Implementation of changes to support the ongoing integration of the toolkit practices into facility workflows; d) Review and interpretation of primary outcomes feedback reports; e) Identification and prioritization of future change efforts; f) Development of a plan to sustain improvement efforts following completion of the study.

C.2.3. Peer-to-Peer Learning Meetings
The research team will host a series of learning meetings during which champions in intervention NHs will be encouraged to discuss successes/challenges and share ideas and strategies they employed to implement the toolkit. Participants will also be encouraged to collaborate with their peers in other facilities outside the formal peer-to-peer meetings.

C.2.4. Enhanced Feedback Reports
Intervention NHs will be able to track their data using the web-based tracking tool that will be accessible by the control NHs. However, the feedback reports made available to intervention NHs will include information on the performance of other participating NHs (in aggregate) allowing the intervention NHs to trend their own data and benchmark themselves against their peers.

D.1. Overview
20 Wisconsin NHs, purposively selected for their geographic location to achieve a balance between rural and urban facilities, will be selected for participation in this study. Our rationale for employing this sampling approach is that rural NHs are less likely to be members of a NH network and may have more limited access to improvement resources and expertise. This approach will enhance our capacity to identify barriers to implementation that will be encountered in NHs throughout Wisconsin and enhance future toolkit dissemination efforts.

D.1.1. Inclusion criteria
Health Sciences IRB Protocol 2018 PI: Christopher Crnich A Cluster Randomized Trial to Assess the Impact of Facilitated Implementation on Antibiotic Stewardship in Wisconsin Nursing Homes Page 5 a) Medicare and Medicaid (dually) certified nursing homes b) Long-term care and skilled nursing beds > 50 beds c) The management of the facility agrees to random allocation to control or intervention group d) The facility is able to submit 3 sequential months of data on facility urine culture and antibiotic treatment rates via the study web portal

D.1.2. Exclusion criteria
a) Assisted living facility wards b) Specialty care (ventilator or strict rehabilitation) wards

D.2. Recruitment
Facilities have been recruited through referrals from members of the Wisconsin Healthcare-Associated Infections in Long-Term Coalition as well as Wisconsin nursing home associations (Leading Age and Wisconsin Health Care Association).

D.3. Randomization
Randomization will be computer generated and stratified based on geographic location (urban vs. rural) in block sizes of two.

E.1. Overview
While the individual components of the Wisconsin UTI Improvement Toolkit are evidence-based, their combined effects on antibiotic prescribing are not known. We will assess the combined effects of the toolkit by comparing UTI treatment rates in study NHs before and after implementation of the toolkit using segmented regression methods. A number of data measures will be employed during this study.

. Primary Outcome Measures
Participating NHs will submit data on the number of urine cultures and antibiotic starts via a web-based portal on a monthly basis.

E.3.2. Secondary Outcome Measures
Participating NHs will submit data on the number of urine cultures and antibiotic starts meeting Wisconsin appropriateness criteria [cite] via a web-based portal on a monthly basis. The Wisconsin appropriateness criteria [cite] require residents to either exhibit symptoms that localize to the urinary tract (e.g., dysuria, frequency) or systemic signs of infection (e.g., fever, hypotension, etc.) in the absence of another obvious source of infection.
The research team will obtain aggregate data on antibiotic prescriptions from the facility consultant pharmacy in order to determine days of antibiotic therapy per 1,000 resident-days as well as the number of fluoroquinolone starts and days of fluoroquinolone antibiotic therapy.
The research team will obtain aggregate data on urine culture results to determine the rates of urine culture resistance in study nursing homes. Clostridium difficile results at a facility level will be obtained in a similar manner.
Facilities will be asked to provider their aggregate resident transfer and mortality data on a monthly basis.

E.3.3. Facility Demographics
A facility demographics survey will be employed to collect information on study NH size, staffing levels, and types of skilled nursing services provided, etc. The Certification and Survey Provider Enhanced Reporting (CASPER) database, a Center for Medicaid and Medicare Services database that provides information about nursing home demographics and selected quality indicators, will be accessed to determine study nursing home staff turnover, aggregate measures of resident complexity and facility F-tag deficiencies (F-tag 329 [inappropriate medication use] and 441 [infection control]). A survey instrument developed by the research team [cite] will be used to characterize each facility's existing antibiotic stewardship structure and process. Another survey instrument developed by the research team [cite] will be employed to assess each facility's organizational culture. We will collect data on the number and the percentage of residents with UTI, as well as on the prevalence of UTI infections and other infections using Health Sciences IRB Protocol 2018 PI: Christopher Crnich A Cluster Randomized Trial to Assess the Impact of Facilitated Implementation on Antibiotic Stewardship in Wisconsin Nursing Homes Page 8 the CHSRA/DHS Nursing Home Scorecard. This scorecard, which is being developed by the UW Center for Health Systems Research and Analysis (CHSRA), provides trends across a wide array of resident level outcomes for Wisconsin Nursing Homes. It also includes comparisons to peer and NHs across the state. The scorecard will be used to report changes in these outcomes.

E.4. Data Analyses
Generalized estimating equations using segmented regression and a Poisson distribution accounting for clustering by facility will be employed to evaluate the two primary outcomes across all study sites in aggregate. Four estimates from these models will be evaluated: 1) the pre-intervention trend ( 1 ); 2) the immediate change in the outcomes following the kickoff webinar ( 2 ); 3) the difference between pre-and post-implementation trend ( 3 ); and 4) the post-implementation outcome rate change ( 1 + 3 ). First and second order autocorrelation will be assessed using the Durbin-Watson statistic. Robust standard errors will be used to estimate variance. Similar generalized linear models will be fit to analyze secondary and safety outcomes. To assess the impact of enhanced implementation on the effectiveness of the toolkit, the segmented regression model developed will be extended to include a covariate for the study implementation arm (control arm = 0; intervention arm = 1). We will explore adding control variables into our models if significant imbalances in NH characteristics (e.g., size, staffing ratios) are identified in our preliminary analyses. Based on data previously collected by our group [6], the expected rate of UTI treatment is ~4.0 per 1,000 resident-days. Assuming implementation of the UTI Toolkit leads to at least a 15% reduction in the UTI treatment rate and the facility dropout rate is <25%, our study will have 90% power to detect a statistically significant difference of this magnitude.

F. Program Evaluation
Please note that the evaluation methods that follow are currently still being developed and the content of this section will likely change. This section will be finalized and IRB protocols will be updated before commencing any the data collection described below.

F.1. Overview
A mixed methods evaluation structured around the RE-AIM framework (Reach, Effectiveness, Adoption, Implementation and Maintenance) [32] will be employed to compare the impact of the two implementation strategies using data submitted by facilities, surveys, interviews and observations. Statistical comparisons using generalized linear mixed models [33] will be employed to examine if differences in antibiotic utilization for treatment of UTI exist between NHs in the two study arms. Data from surveys, interviews and observations will be analyzed using a mixed-methods approach to ascertain the extent to which recommended toolkit practices have been implemented in study NHs.

F.2.1. Nursing Home Residents
As the intervention in this study is targeted at the organizational level and not the individual resident, we will not be recruiting residents as part of this study. In order to assess the impact of the UTI Improvement Toolkit on antibiotic prescribing, research staff will review the aggregate data about residents who have been treated for a UTI before and after the intervention to determine if documentation reflects a change in management practices in the facility. The resident data that will be collected are listed in the table in section E.1, under "Resident Level Outcomes".

F.2.2. Nursing Staff and Providers
Health Sciences IRB Protocol 2018 PI: Christopher Crnich A Cluster Randomized Trial to Assess the Impact of Facilitated Implementation on Antibiotic Stewardship in Wisconsin Nursing Homes Page 9 We will be recruiting nursing staff and providers in order to evaluate facility engagement in the implementation of the toolkit and the factors that influence implementation.

F.3. Subject Recruitment
Within each nursing home, the Practice Manager will be contacted to identify staff to participate in interviews. These individuals will be sent an email (developed by the research team but sent by their practice manager) that invites them to participate and lists a telephone number to call the research team about questions or concerns. An information flyer (i.e., recruitment flyer) will also be attached to the email. If any of the clinician staff do not use an individual email account for work, the email text will be sent as a letter to those potential participants by postal mail, with a hard copy of the information flyer enclosed.
For the interviews and observations, all facility-employed and per diem nursing staff (certified nursing assistants [CNA], licensed practical nurses [LPN] and registered nurses [RN]) who are responsible for the care of >3 residents will be included in the study. For the questionnaires, all nursing home clinical care staff will be invited to participate.

F.4.
Data Collection ProceduresWe will collect data on each part of the RE-AIM framework: Reach, Effectiveness, Adoption, Implementation and Maintenance.

F.4.1. Reach
By means of interviews and surveys, we will collect data on the number of NHs that adopt the UTI Toolkit with fidelity. The surveys and the interview guides will be submitted for IRB approval via a change in protocol when developed at the end of Year 1.

F.4.2. Effectiveness
Some of the data mentioned above will also be used to assess Effectiveness. Specifically, the data on Nursing Home Quality Indicators and on Resident Level Outcomes (see Table in Section III. E.1) will be used here.

F.4.3. Adoption
We will collect data on the knowledge, attitudes, and perceptions of NH staff about the UTI toolkit. We will also use some of the data mentioned above, namely the data on Organizational Culture (see Table in Section III. E.1).

F.4.4. Implementation
To assess implementation, we will use a UTI toolkit fidelity checklist. We will also collect data to identify barriers to, and facilitators, of implementation. To collect these data, in addition to interviews and surveys, we will use a coaching workbook, on which the clinical coach will track their interactions with NH staff. Characteristics of these interactions, including length of encounters, number of participants, notable facility developments (e.g., leadership turnover, survey visits), progress made, barriers encountered and suggested strategies for overcoming identified barriers, will be detailed in this workbook.

F.4.5. Maintenance
Maintenance will be assessed by evaluating if participating nursing homes were able to sustain over a 9 month period the Nursing Home Quality Indicators and the Resident Level Outcomes (see Table in Section Health Sciences IRB Protocol 2018 PI: Christopher Crnich A Cluster Randomized Trial to Assess the Impact of Facilitated Implementation on Antibiotic Stewardship in Wisconsin Nursing Homes Page 10 III. E.1). We will also utilize qualitative interviews with staff (guides will be submitted for IRB approval via a change in protocol when developed at the end of Year 1) to assess facilitators and barriers to sustainability of the intervention in the NHs.

F.5. Data Analyses
A mixed methods evaluation approach structured around the RE-AIM framework [32] will be employed in this aim. Reach is the percentage of NHs that adopt the Wisconsin UTI Improvement Toolkit with fidelity. To compare differences in toolkit effectiveness between the two implementation strategies, the segmented regression model that will be developed (see Section III.E.4) will be extended to include a covariate for the study implementation arm (control = 0; externally-facilitated = 1). Adoption will be assessed both quantitatively (differences between NHs in the two implementation arms will be assessed by survey items measuring organizational readiness for change, staff knowledge, attitudes and perceptions about the UTI toolkit) and qualitatively using a thematic approach [34] structured around System Engineering Initiative for Patient Safety (SEIPS) framework [35]. Differences in the extent of implementation in NHs randomized to the two approaches will be assessed by comparing responses to a UTI toolkit fidelity checklist and qualitatively through analyses of data collected during staff interviews that focus on the identification of implementation strategies and barriers. Short-term maintenance of the effectiveness outcomes will be assessed utilizing an approach developed by Drs. Ford and Robinson [36].
To assess the impact of coaching, we will utilize the comparative conceptual framework proposed by Bond and Seneque [37] which suggests that organizational coaching occurs across one of five domains: managerial, consulting, mentoring, facilitating or coaching to classify the advice provided during the coaching interactions. Quantitative and qualitative data will be integrated in an attempt to explain implementation success and failures using a formal triangulation approach [38]. Coaching notes, interview transcripts with NH champions, and notes generated during hosted peer-to-peer learning sessions will be analyzed thematically to identify common barriers NHs encountered and strategies employed to overcome them as well as the types of coaching instruction that were most impactful on facility implementation efforts.

F.6.1. Interviews
No identifiable information about the interview participants will intentionally be gathered. If any such information is mentioned during the interview, it will be removed from the audio file and transcript as soon as it is noticed by the research team. Interviews will be transcribed by a UW-Madison approved transcription service, Premium Business Services, which uses a HIPAA-compliant secure server to transfer files. Transcriptions will be reviewed and redacted for unintentional mention of personal health information or nursing home identifiers.
Audio files and transcripts will be stored on a secure password-protected server maintained by the Department of Medicine in the UW School of Medicine and Public Health. Only UW research team members will have access to these files through password-protected computers. Cleaned and de-identified transcripts may be stored on the local hard drives of the password-protected computers of UW research team members while the data are being analyzed.

F.6.2. Observations
The observations will be completed by clinical coach, who will not record any identifying information about the observation participant or any information about patients in their notes. Field notes will also be redacted for nursing home identifiers. Field notes will not contain PHI, as they relate to facility-level systems, which do not require patient-specific information.
Health Sciences IRB Protocol 2018 PI: Christopher Crnich A Cluster Randomized Trial to Assess the Impact of Facilitated Implementation on Antibiotic Stewardship in Wisconsin Nursing Homes Page 11 The notes will be typed up by the clinical coach, carefully checked to remove any identifying information, and stored electronically. Each coach will retain a personal copy of their notes, which they will refer to in answering questions during data analysis. These files will be stored in a secure location convenient to the coach on a password-protected computer maintained by the Department of Medicine in the UW School of Medicine and Public Health. All clinical coaches will have access to their own notes, but only UW research team members will have access to the electronic notes of the entire research team, which will be stored on a secure password-protected server maintained by the Department of Medicine in the UW School of Medicine and Public Health. The coaches will destroy personal copies of their notes when the study is completed.
To protect identifiable subject information, we will store a list of contact information for clinic staff on a secure password protected server maintained by the Department of Medicine in the UW School of Medicine and Public Health. The list will only be accessible through password-protected computers of the UW researchers. The contact information will not contain any links to data and will be destroyed after the study is completed. All email correspondence with participants, both sent and received, will also be destroyed after the study is completed.
The audio files, de-identified transcripts and all other data will be destroyed after seven years, in compliance with UW's recommended retention period.

F.6.3. Surveys
As previously explained, we will use the UW Antibiotic Stewardship Program (ASP) survey, the UW organizational survey, the CHSRA/DHS Nursing Home Scorecard, and a survey to collect data on nursing homes that adopt the UTI Toolkit with fidelity and to identify barriers to, and facilitators of, implementation. We plan to either mail these surveys to the staff of the participating facilities or have them complete the surveys electronically through REDCap (ICTR instance). The surveys will be anonymous, and, for paper surveys, the participants will be asked to mail them to the research team by using a postage-paid return envelope that we will include with the surveys.
The completed surveys will be stored in a locked cabinet in the Department of Medicine in the UW School of Medicine and Public Health. Only UW research team members will have access to these documents on REDCap. The documents will be destroyed after seven years, in compliance with UW's recommended retention period.