Evaluation of an Educational Outreach and Audit and Feedback Program to Reduce Continuous Pulse Oximetry Use in Hospitalized Infants With Stable Bronchiolitis

Key Points Question Are audit and feedback strategies and educational outreach associated with clinician perceptions of the feasibility, acceptability, appropriateness, and safety of continuous pulse oximetry use in children hospitalized with bronchiolitis who are not receiving supplemental oxygen (guideline-discordant use)? Findings In this 6-hospital single-group nonrandomized clinical trial, 847 nurses and physicians highly rated the feasibility, acceptability, and appropriateness of audit and feedback strategies and educational outreach. Guideline-discordant pulse oximetry use decreased from 53% to 23% during the intervention period. Meaning Educational outreach and audit and feedback strategies were feasible, acceptable, appropriate, and associated with a reduction in guideline-discordant continuous pulse oximetry use in children hospitalized with bronchiolitis who are not receiving supplemental oxygen.


Introduction
Bronchiolitis is the leading reason for non-birth-related hospitalization in infants, accounting for more than 100 000 hospitalizations and $734 million in hospital costs annually. 1 Bronchiolitis is a self-limited viral illness with a well-documented clinical course and national evidence-based practice guidelines. [2][3][4] In addition to management recommendations, guidelines advise against continuous pulse oximetry monitoring (cSpO 2 ) for patients who are not receiving supplemental oxygen, as it increases health care use without improving outcomes. 2,3,[5][6][7][8][9] Although none of the guidelines that discourage cSpO 2 explicitly define it, we consider any use of pulse oximetry beyond a spot check (an in-person assessment of the oxygen saturation, with the probe applied and removed by staff during a single visit to the patient's room) to represent continuous measurement. A recent study measured cSpO 2 use at 56 US and Canadian hospitals and found that, overall, 46% of patients received cSpO 2 , suggesting a gap between the evidence-based guidelines and real-world clinical practice. 10 Implementation science seeks to overcome evidence-to-practice gaps. 11,12 Audit and feedback and educational outreach are common implementation strategies that are effective in improving processes of care and clinical outcomes 13 and are used frequently and successfully in quality improvement initiatives in pediatric hospital settings. [14][15][16][17] However, the feasibility of audit and feedback and educational outreach and their associations with successful deimplementation in the pediatric hospital setting is limited mainly to quality improvement collaboratives, with diverse bundled interventions and multiple practices targeted for deimplementation. 18,19 In this study, our objective was to measure the feasibility, acceptability, appropriateness, and

Design
This was a prospective, nonrandomized, single-group implementation feasibility trial with historical control data from the baseline period listed above. We invited hospitals that participated in the baseline study to express interest in pilot trial participation using an online form. Twenty-three hospitals expressed interest. In addition to the primary study site (Children's Hospital of Philadelphia), we invited 5 sites with risk-standardized monitoring percentages of 60% or more during the baseline study aiming to optimize diversity of geography and community hospital participation, as well as availability of monitor data in the electronic health record for a related substudy. 20 The intervention consisted of 2 deimplementation strategies: educational outreach and audit and feedback (Figure 1). These strategies were chosen based on barriers and facilitators identified in prior qualitative research 21 and were then mapped to deimplementation strategies during 2 stakeholder strategy development panels. The education and audit and feedback interventions were delivered in person to physicians, nurses, and respiratory therapists.

Educational Outreach Intervention
Educational outreach started before audit and feedback, then continued at a frequency determined by the site principal investigator based on study site needs. Education included 3 components: national guidelines for cSpO 2 monitoring in patients with bronchiolitis, 2-4 evidence supporting intermittent pulse oximetry rather than cSpO 2 monitoring, and their hospital's baseline cSpO 2 use 10 (eFigure 1 in Supplement 2). Site principal investigators tailored non-core content (eg, logos and location-specific context) and the setting of sessions. Sites were asked to target a 50% reduction in cSpO 2 use.

Audit and Feedback Intervention and cSpO 2 Use Data Collection
Study team members underwent webinar-based training in the fall of 2019. During the intervention period, local study teams conducted medical record review to screen patients for eligibility, followed by bedside data collection rounds using previously published methods 10,22 to determine the cSpO 2 Timeline depicting the intervention including data collection, audit and feedback (A&F), and educational outreach interventions. Audit and feedback intervention consisted of bedside data collection on continuous pulse oximetry monitoring status (the audit) and 2 forms of feedback: individual real-time inquiry conducted at the time of data collection and weekly unit-level performance feedback. Educational outreach intervention consisted of the 3 core components, which included national guidelines for pulse oximetry monitoring in patients with bronchiolitis, evidence supporting intermittent rather than continuous pulse oximetry monitoring, and the hospital's baseline and current pulse oximetry monitoring performance. status of patients with bronchiolitis not receiving supplemental oxygen. These data served as the outcome measure for cSpO 2 use and the audit data for audit and feedback. Teams were encouraged to conduct data collection twice weekly, with timing based on availability of the data collector. Given that rapid improvements in clinical status are expected with bronchiolitis 23 and that cSpO 2 use should change accordingly, sites could collect data from the same patient on multiple occasions if observations were more than 6 hours apart. An identifier allowed accounting for clustering in analyses. Eligible patients were between the ages of 8 weeks and 23 months and admitted to a general medical service with a primary diagnosis of bronchiolitis. We excluded patients documented as premature or preterm and those with documented prematurity of less than 28 weeks' gestation; cyanotic congenital heart disease or pulmonary hypertension; home oxygen use, positive pressure ventilation requirement, or tracheostomy; primary neuromuscular disease; immunodeficiency; or cancer. Additional exclusions were added in December 2019 (heart failure, myocarditis, or arrhythmia) and in March 2020 (COVID-19).
The feedback intervention had 2 forms: individual real-time inquiry and weekly unit-level feedback. In real-time inquiry, if clinicians were available during cSpO 2 audits, data collectors asked briefly and nonjudgmentally about the indication for cSpO 2 . In unit-level feedback, the study coordinating team summarized each participating unit's data in a weekly unit-specific dashboard (an electronic document with unit-specific cSpO 2 monitoring performance and reminders for specific practices to improve performance) sent to site principal investigators that reiterated educational outreach information (eFigure 2 in Supplement 2). In the dashboard, which site principal investigators tailored to site needs, data were compared with the performance of other hospital units, the hospital's baseline performance, 10 and the hospital's target performance. The study coordinating team also suggested improvement targets based on local monitoring patterns, such as day and night variation.

Outcomes
The primary outcomes were acceptability, appropriateness, and feasibility of the deimplementation strategies. 21 We also measured perceived safety of intermittently spot-checking oxygen saturation instead of using cSpO 2 . To estimate penetration 24 of guideline-concordant care 2-4 of patients with bronchiolitis not receiving supplemental oxygen, we assessed the change in cSpO 2 use between the baseline 10 and intervention periods.

Implementation Outcomes
Aiming to distribute a brief instrument with minimal overlap between questions, our multidisciplinary study team of experts in pediatrics, nursing, clinical research, patient safety, and implementation science parsimoniously selected and adapted items from the validated Acceptability of Intervention Measure, Intervention Appropriateness Measure, and Feasibility of Intervention Measure instruments 25 for the study questionnaire. The team also developed questionnaire items focused on perceived safety, norms, and intentions using published guidance on constructing Theory of Planned Behavior-based questionnaires. 26 The resulting questionnaire (eAppendix in Supplement 2) was distributed electronically to study unit clinicians, including nurses, advanced practice nurses, and resident, fellow, and attending physicians. Respiratory therapists were not included because their scope of practice does not include managing cSpO 2 monitoring and their coverage is spread across multiple hospital units, making it difficult to identify individuals who worked on intervention units. Clinicians who reported caring for patients with bronchiolitis on intervention units but not being exposed to either intervention only completed the questions about perceived safety, norms, and intentions.

Clinical Outcomes
We report the percentage of patients with bronchiolitis not receiving supplemental oxygen who were receiving cSpO 2 as "guideline-discordant monitoring" in the baseline and intervention periods. To report the estimated change in penetration 24,27 of guideline-concordant care (avoiding cSpO 2 for patients not receiving supplemental oxygen), we examined the reduction in cSpO 2 between the baseline and intervention periods.

JAMA Network Open | Pediatrics
Patient characteristics were abstracted from the electronic health record, including age, history of prematurity, sex, race, ethnicity, time since weaning from supplemental oxygen, history of apnea or a condition associated with neurologic impairment (eg, cerebral palsy), and presence of an enteral feeding tube.

Adverse Event Surveillance
We performed active surveillance for adverse events that could be associated with reductions in cSpO 2 . Staff at each site screened locally available data for code blue and rapid response team activations in any patients with bronchiolitis hospitalized on study units. Medical records of patients meeting these criteria were reviewed, and staff involved in the event were interviewed if necessary.
If the patient was unmonitored during the event, there was further investigation. Events were considered at least possibly related to the study intervention if there was "a reasonable possibility that the adverse event may have been caused by the procedures involved in the research." 28(p1) Events determined to be at least possibly related to the study intervention were escalated according to local institutional review board protocols.

Statistical Analysis
For the questionnaire-based outcomes, we first summarized responses to each question descriptively. We then explored differences in responses between nurses and physicians using Pearson χ 2 tests. We used ordinal logistic regression accounting for hospital-level clustering and reported odds ratios (ORs) with 95% CIs for nurses, with physicians as the reference group. The OR in ordinal logistic regression indicates the odds of choosing a response on the Likert scale 1 unit higher in agreement vs a response less than or equal to that level. 29 Because these ORs can be difficult to interpret, for questions with significant differences in agreement between nurses and physicians, we calculated predictive marginal probabilities of each level of agreement 29 and compared them by profession. We did not make any adjustments for multiple comparisons. 30 We calculated unadjusted guideline-discordant monitoring percentages for each hospital using the denominator of all directly observed patients with bronchiolitis who were not receiving supplemental oxygen and the numerator of patients who were simultaneously receiving cSpO 2 . We further compared baseline and intervention period data overall and at the hospital level using logistic regression, adjusted for the same covariates used in previous research, including age combined with preterm birth, time since weaning from supplemental oxygen, documented history of apnea or cyanosis during the present illness, presence of an enteral feeding tube, neurological impairment, and nighttime observation. 10 To obtain adjusted estimates at the hospital level, we included an interaction term between hospital and the intervention period. The model accounted for clustering of observations within patient admissions.
To report the estimated change in penetration 24 Data Capture and hosted at Children's Hospital of Philadelphia. 31 We used Stata, version 16.0

Results
The We distributed questionnaires to 1263 clinicians; 1193 clinicians were eligible and 847 responded, for an overall 71% response rate (range, 66%-84% among hospitals) (Figure 2). Of the 847 respondents, 474 reported attending at least 1 educational session, and 664 reported being provided with feedback data about their unit's performance at least once; those respondents completed the corresponding questionnaire regarding acceptability, appropriateness, and feasibility.
Additional details on response rates by profession are presented in eTable 1 in Supplement 2. Results are summarized in Table 1 and eTable 2 in Supplement 2.

Educational Sessions
Respondents rated educational sessions favorably. Most agreed or completely agreed that they liked

Audit and Feedback
Respondents also rated the audit and feedback intervention favorably (eTable 2 in Supplement 2).

Most agreed or completely agreed that they liked (615 of 664 [93%]) and welcomed (636 of 664
[96%]) the intervention (acceptability) and that the intervention was appropriate (622 of 664 [94%]) and feasible (557 of 664 [84%]). There were no significant differences between physician and nurse responses to questions about audit and feedback feasibility (Table 1). However, nurses had lower odds than physicians of agreeing that they welcomed continued feedback (a measure of acceptability; OR, 0.57; 95% CI, 0.33-1.00, P = .048). of 847) agreed that intermittently spot-checking oxygen saturation instead of cSpO 2 put patients at risk, and only 15% (129 of 847) agreed that intermittently spot-checking oxygen saturation instead of cSpO 2 was upsetting to parents. Compared with physicians, nurses had lower odds of agreeing that intermittently spot-checking oxygen saturation instead of cSpO 2 monitoring is safe (OR, 0.28; 95% CI 0.24-0.33; P < .001) and that it is a good idea (OR, 0.28; 95% CI, 0.24-0.32; P < .001). These interprofessional contrasts were driven by differences in responses of "completely agree" vs "agree" (eTable 2 in Supplement 2). Nurses had significantly higher odds of agreeing that intermittently spot-checking oxygen saturation instead of cSpO 2 is upsetting to parents (OR, 2.38, 95% CI,

Clinical Monitoring Use
Patient characteristics for the baseline and intervention periods are presented in Table 2. comparing unit performance between the baseline and intervention periods was not possible. Using data from baseline period hospital units and adjusting for the same covariates used in the observational study's analysis, 10 guideline-discordant cSpO 2 use decreased from 53% (95% CI, 49%-57%) to 23% (95% CI, 20%-25%; P < .001) during the intervention period ( Table 3). The cSpO 2 prevalence from the intervention period was 31 percentage points lower (95% CI, 26-35 percentage points) compared with baseline. This equates to a 31-percentage point increase in penetration 24,27 of guideline-concordant care. There were no adverse events attributable to the intervention during the study.

Discussion
In this 6-hospital single-group cSpO 2 deimplementation trial using historical control data, most respondents agreed that the deimplementation strategies targeting guideline-discordant cSpO 2 for patients with bronchiolitis not receiving supplemental oxygen were acceptable, appropriate, feasible, and safe. Application of these strategies was temporally associated with a significant decrease in the adjusted percentage of hospitalized children with bronchiolitis not receiving supplemental oxygen who received guideline-discordant cSpO 2 .
We noted important interprofessional differences in perceptions of intervention safety between nurses and physicians, which warrant further study and have implications for future deimplementation efforts. The deimplementation of practices considered safe in children may be particularly challenging. 32 Continuous physiological monitoring has been widely adopted into clinical surveillance in various settings based on a common belief that it improves safety. 33 Nurses in adult settings report that continuous physiological monitoring of patients outside of the intensive care unit improves patient safety. 34 We found significant differences in the perceived safety of intermittently spot-checking oxygen saturation, with nurses rating this approach significantly lower than physician participants. The contrast was driven by differences in the distributions of "completely agree" vs "agree" responses, however, which may or may not be clinically important. This observed difference may relate to the scope of practice for nurses in pediatric hospital settings, where advocating for clinical decisions regarding the escalation or de-escalation of physiological monitoring to improve the detection of clinical deterioration is common. 35 Furthermore, the education and audit and feedback interventions were delivered to each professional in the same format in this study. It is possible that tailoring the audit and feedback strategy to include clinicians setting role-specific goals, providing clinician-concordant benchmarks, and ensuring role concordance of the person delivering the feedback may improve perceptions of safety. Although clinicians overall rated the intervention favorably, the differences we observed in nurses' perception of safety will need to be a focus of future deimplementation work in this area.
Although our findings suggest that audit and feedback and education are associated with positive clinician perceptions and deimplementation of cSpO 2 , prior studies suggest that decay in improvement can occur after they are removed. 36 A follow-up study of a multicenter learning collaborative found that many interventions were not sustained after the intervention period ended. 37,38 Educational outreach strategies declined in use from 73% to 37% of hospitals, and data audits declined from 88% to 30%, with respondents citing insufficient time and competing priorities. These findings are consistent with the perspective that education-based interventions are a necessary component in multistrategy interventions but rarely result in sustained behavior changes alone. 39 Education may require fewer resources to implement than audit and feedback, as suggested by the high feasibility ratings in our study. Future studies can elucidate mechanisms that contribute to the effectiveness of differing configurations, including the ideal "dose" of audit and feedback and educational outreach, and anticipate necessary adaptations depending on the setting.
The specific aspects of the education and audit and feedback strategies' association with cSpO 2 deimplementation in children's hospitals are not well-established. Three single-center quality improvement studies used audit and feedback or educational outreach alongside additional components, such as the use of champions, standard pathways, and order set modifications. 14,16,17 A multicenter improvement study using education along with capacity building saw increases in intermittent pulse oximetry orders. 15 Although these studies describe improvements in practice, most used multiple concurrent or sequential interventions with less focus on evaluating distinct strategies. Based on the generally favorable clinician perceptions, our findings support further testing of audit and feedback and educational outreach to address cSpO 2 use for hospitalized patients with bronchiolitis. Additional interventions, such as champions and electronic health record-based interventions, should be assessed in future studies to determine whether they are associated with deimplementation.

Limitations
This study has several limitations. First, 5 of the 6 study hospitals were freestanding children's hospitals, and all participated in the baseline measurement study 10 in which some clinicians may have been exposed to education. Our findings may not be generalizable to all inpatient settings