[Skip to Navigation]
Sign In
Figure 1.  Study Timeline
Study Timeline

Timeline depicting the intervention including data collection, audit and feedback (A&F), and educational outreach interventions. Audit and feedback intervention consisted of bedside data collection on continuous pulse oximetry monitoring status (the audit) and 2 forms of feedback: individual real-time inquiry conducted at the time of data collection and weekly unit-level performance feedback. Educational outreach intervention consisted of the 3 core components, which included national guidelines for pulse oximetry monitoring in patients with bronchiolitis, evidence supporting intermittent rather than continuous pulse oximetry monitoring, and the hospital’s baseline and current pulse oximetry monitoring performance.

Figure 2.  Questionnaire Participants
Questionnaire Participants

Clinicians invited to participate included nurses, physicians, and respiratory therapists.

Table 1.  Results of Ordered Logistic Regression Model: Nurse and Physician Respondents
Results of Ordered Logistic Regression Model: Nurse and Physician Respondents
Table 2.  Patient Characteristics
Patient Characteristics
Table 3.  Continuous Spo2 Use Rates in Patients Not Receiving Supplemental Oxygen (Guideline-Discordant Monitoring)
Continuous Spo2 Use Rates in Patients Not Receiving Supplemental Oxygen (Guideline-Discordant Monitoring)
1.
Fujiogi  M, Goto  T, Yasunaga  H,  et al.  Trends in bronchiolitis hospitalizations in the United States: 2000-2016.   Pediatrics. 2019;144(6):e20192614. doi:10.1542/peds.2019-2614 PubMedGoogle Scholar
2.
Ralston  SL, Lieberthal  AS, Meissner  HC,  et al; American Academy of Pediatrics.  Clinical practice guideline: the diagnosis, management, and prevention of bronchiolitis.   Pediatrics. 2014;134(5):e1474-e1502. doi:10.1542/peds.2014-2742 PubMedGoogle ScholarCrossref
3.
Quinonez  RA, Garber  MD, Schroeder  AR,  et al.  Choosing wisely in pediatric hospital medicine: five opportunities for improved healthcare value.   J Hosp Med. 2013;8(9):479-485. doi:10.1002/jhm.2064 PubMedGoogle ScholarCrossref
4.
Schondelmeyer  AC, Dewan  ML, Brady  PW,  et al.  Cardiorespiratory and pulse oximetry monitoring in hospitalized children: a Delphi process.   Pediatrics. 2020;146(2):e20193336. doi:10.1542/peds.2019-3336 PubMedGoogle Scholar
5.
Cunningham  S, Rodriguez  A, Adams  T,  et al; Bronchiolitis of Infancy Discharge Study (BIDS) group.  Oxygen saturation targets in infants with bronchiolitis (BIDS): a double-blind, randomised, equivalence trial.   Lancet. 2015;386(9998):1041-1048. doi:10.1016/S0140-6736(15)00163-4 PubMedGoogle ScholarCrossref
6.
McCulloh  R, Koster  M, Ralston  S,  et al.  Use of intermittent vs continuous pulse oximetry for nonhypoxemic infants and young children hospitalized for bronchiolitis: a randomized clinical trial.   JAMA Pediatr. 2015;169(10):898-904. doi:10.1001/jamapediatrics.2015.1746 PubMedGoogle ScholarCrossref
7.
Principi  T, Coates  AL, Parkin  PC, Stephens  D, DaSilva  Z, Schuh  S.  Effect of oxygen desaturations on subsequent medical visits in infants discharged from the emergency department with bronchiolitis.   JAMA Pediatr. 2016;170(6):602-608. doi:10.1001/jamapediatrics.2016.0114 PubMedGoogle ScholarCrossref
8.
Schuh  S, Freedman  S, Coates  A,  et al.  Effect of oximetry on hospitalization in bronchiolitis: a randomized clinical trial.   JAMA. 2014;312(7):712-718. doi:10.1001/jama.2014.8637 PubMedGoogle ScholarCrossref
9.
Schuh  S, Kwong  JC, Holder  L, Graves  E, Macdonald  EM, Finkelstein  Y. Predictors of critical care and mortality in bronchiolitis after emergency department discharge. J Pediatr. 2018;199:217-222.e1. doi:10.1016/j.jpeds.2018.04.010
10.
Bonafide  CP, Xiao  R, Brady  PW,  et al; Pediatric Research in Inpatient Settings (PRIS) Network.  Prevalence of continuous pulse oximetry monitoring in hospitalized children with bronchiolitis not requiring supplemental oxygen.   JAMA. 2020;323(15):1467-1477. doi:10.1001/jama.2020.2998 PubMedGoogle ScholarCrossref
11.
Eccles  MP, Mittman  BS.  Welcome to Implementation Science.   Implement Sci. 2006;1:1. doi:10.1186/1748-5908-1-1 Google ScholarCrossref
12.
Bauer  MS, Damschroder  L, Hagedorn  H, Smith  J, Kilbourne  AM.  An introduction to implementation science for the non-specialist.   BMC Psychol. 2015;3(1):32. doi:10.1186/s40359-015-0089-9 PubMedGoogle ScholarCrossref
13.
Jamtvedt  G, Young  JM, Kristoffersen  DT, O’Brien  MA, Oxman  AD.  Does telling people what they have been doing change what they do? a systematic review of the effects of audit and feedback.   Qual Saf Health Care. 2006;15(6):433-436. doi:10.1136/qshc.2006.018549 PubMedGoogle ScholarCrossref
14.
Schondelmeyer  AC, Simmons  JM, Statile  AM,  et al.  Using quality improvement to reduce continuous pulse oximetry use in children with wheezing.   Pediatrics. 2015;135(4):e1044-e1051. doi:10.1542/peds.2014-2295 PubMedGoogle ScholarCrossref
15.
Ralston  SL, Garber  MD, Rice-Conboy  E,  et al; Value in Inpatient Pediatrics Network Quality Collaborative for Improving Hospital Compliance with AAP Bronchiolitis Guideline (BQIP).  A multicenter collaborative to reduce unnecessary care in inpatient bronchiolitis.   Pediatrics. 2016;137(1). doi:10.1542/peds.2015-0851 PubMedGoogle Scholar
16.
Heneghan  M, Hart  J, Dewan  M,  et al.  No cause for alarm: decreasing inappropriate pulse oximetry use in bronchiolitis.   Hosp Pediatr. Published online January 30, 2018. doi:10.1542/hpeds.2017-0126 PubMedGoogle Scholar
17.
Mittal  V, Darnell  C, Walsh  B,  et al.  Inpatient bronchiolitis guideline implementation and resource utilization.   Pediatrics. 2014;133(3):e730-e737. doi:10.1542/peds.2013-2881 PubMedGoogle ScholarCrossref
18.
Ralston  SL, Garber  MD, Rice-Conboy  E,  et al. A multicenter collaborative to reduce unnecessary care in inpatient bronchiolitis. Pediatrics. 2016;137(1):e20150851. doi:10.1542/peds.2015-0851
19.
Kaiser  SV, Jennings  B, Rodean  J,  et al. Pathways for Improving Inpatient Pediatric Asthma Care (PIPA): a multicenter, national study. Pediatrics. 2016;145(6):e20193026. doi:10.1542/peds.2019-3026
20.
Rasooly  IR, Kern-Goldberger  AS, Xiao  R,  et al.  Physiologic monitor alarm burden and nurses’ subjective workload in a children’s hospital.   Hosp Pediatr. 2021;hpeds.2020-003509. doi:10.1542/hpeds.2020-003509PubMedGoogle Scholar
21.
Wolk  CB, Schondelmeyer  AC, Barg  FK,  et al; Pediatric Research in Inpatient Settings (PRIS) Network.  Barriers and facilitators to guideline-adherent pulse oximetry use in bronchiolitis.   J Hosp Med. 2021;16(1):23-30. doi:10.12788/jhm.3535 PubMedGoogle ScholarCrossref
22.
Rasooly  IR, Beidas  RS, Wolk  CB,  et al; Pediatric Research in Inpatient Settings (PRIS) Network.  Measuring overuse of continuous pulse oximetry in bronchiolitis and developing strategies for large-scale deimplementation: study protocol for a feasibility trial.   Pilot Feasibility Stud. 2019;5:68. doi:10.1186/s40814-019-0453-2 PubMedGoogle ScholarCrossref
23.
Mansbach  JM, Clark  S, Piedra  PA,  et al; MARC-30 Investigators.  Hospital course and discharge criteria for children hospitalized with bronchiolitis.   J Hosp Med. 2015;10(4):205-211. doi:10.1002/jhm.2318 PubMedGoogle ScholarCrossref
24.
Prusaczyk  B, Swindle  T, Curran  G.  Defining and conceptualizing outcomes for de-implementation: key distinctions from implementation outcomes.   Implement Sci Commun. 2020;1:43. doi:10.1186/s43058-020-00035-3 PubMedGoogle ScholarCrossref
25.
Weiner  BJ, Lewis  CC, Stanick  C,  et al.  Psychometric assessment of three newly developed implementation outcome measures.   Implement Sci. 2017;12(1):108. doi:10.1186/s13012-017-0635-3 PubMedGoogle ScholarCrossref
26.
Ajzen  I. Constructing a theory of planned behavior questionnaire. Published 2006. Accessed November 3, 2020. http://people.umass.edu/~aizen/pdf/tpb.measurement.pdf
27.
Proctor  E, Silmere  H, Raghavan  R,  et al.  Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda.   Adm Policy Ment Health. 2011;38(2):65-76. doi:10.1007/s10488-010-0319-7 PubMedGoogle ScholarCrossref
28.
Committee for the Protection of Human Subjects, The Children’s Hospital of Philadelphia. Unanticipated problems involving risks to subjects. Revised February 25, 2013. Accessed July 26, 2021. https://irb.research.chop.edu/sites/default/files/documents/irbsop408.pdf
29.
Mitchell  MN. Ordinal logistic regression. In:  Interpreting and Visualizing Regression Models Using Stata. Second ed. Stata Press; 2020.
30.
Rothman  KJ.  No adjustments are needed for multiple comparisons.   Epidemiology. 1990;1(1):43-46. doi:10.1097/00001648-199001000-00010 PubMedGoogle ScholarCrossref
31.
Harris  PA, Taylor  R, Thielke  R, Payne  J, Gonzalez  N, Conde  JG.  Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support.   J Biomed Inform. 2009;42(2):377-381. doi:10.1016/j.jbi.2008.08.010 PubMedGoogle ScholarCrossref
32.
Hoffman  JM, Keeling  NJ, Forrest  CB,  et al.  Priorities for pediatric patient safety research.   Pediatrics. 2019;143(2):e20180496. doi:10.1542/peds.2018-0496 PubMedGoogle Scholar
33.
Downey  CL, Chapman  S, Randell  R, Brown  JM, Jayne  DG.  The impact of continuous versus intermittent vital signs monitoring in hospitals: a systematic review and narrative synthesis.   Int J Nurs Stud. 2018;84:19-27. doi:10.1016/j.ijnurstu.2018.04.013 PubMedGoogle ScholarCrossref
34.
Watkins  T, Whisman  L, Booker  P.  Nursing assessment of continuous vital sign surveillance to improve patient safety on the medical/surgical unit.   J Clin Nurs. 2016;25(1-2):278-281. doi:10.1111/jocn.13102 PubMedGoogle ScholarCrossref
35.
Stotts  JR, Lyndon  A, Chan  GK, Bekmezian  A, Rehm  RS.  Nursing surveillance for deterioration in pediatric patients: an integrative review.   J Pediatr Nurs. 2020;50:59-74. doi:10.1016/j.pedn.2019.10.008 PubMedGoogle ScholarCrossref
36.
Gerber  JS, Prasad  PA, Fiks  AG,  et al.  Durability of benefits of an outpatient antimicrobial stewardship intervention after discontinuation of audit and feedback.   JAMA. 2014;312(23):2569-2570. doi:10.1001/jama.2014.14042 PubMedGoogle ScholarCrossref
37.
Biondi  EA, McCulloh  R, Staggs  VS,  et al; American Academy of Pediatrics’ REVISE Collaborative.  Reducing Variability in the Infant Sepsis Evaluation (REVISE): a national quality initiative.   Pediatrics. 2019;144(3):e20182201. doi:10.1542/peds.2018-2201 PubMedGoogle Scholar
38.
Kaiser  SV, Shadman  KA, Biondi  EA, McCulloh  RJ.  Feasible strategies for sustaining guideline adherence: cross-sectional analysis of a national collaborative.   Hosp Pediatr. 2019;9(11):903-908. doi:10.1542/hpeds.2019-0152 PubMedGoogle ScholarCrossref
39.
Soong  C, Shojania  KG.  Education as a low-value improvement intervention: often necessary but rarely sufficient.   BMJ Qual Saf. 2020;29(5):353-357. doi:10.1136/bmjqs-2019-010411 PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Original Investigation
    Pediatrics
    September 2, 2021

    Evaluation of an Educational Outreach and Audit and Feedback Program to Reduce Continuous Pulse Oximetry Use in Hospitalized Infants With Stable Bronchiolitis: A Nonrandomized Clinical Trial

    Author Affiliations
    • 1Department of Systems, Populations, and Leadership, University of Michigan School of Nursing, Ann Arbor
    • 2Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine at the University of Pennsylvania, Philadelphia
    • 3Department of Psychiatry, Perelman School of Medicine at the University of Pennsylvania, Philadelphia
    • 4Department of Medical Ethics and Health Policy, Perelman School of Medicine at the University of Pennsylvania, Philadelphia
    • 5Department of Medicine, Perelman School of Medicine at the University of Pennsylvania, Philadelphia
    • 6Penn Implementation Science Center at the Leonard Davis Institute of Health Economics (PISCE@LDI), University of Pennsylvania, Philadelphia
    • 7Division of General Pediatrics, Department of Pediatrics, Boston Children’s Hospital, Boston, Massachusetts
    • 8Division of Sleep and Circadian Disorders, Departments of Medicine, Brigham and Women’s Hospital, Boston, Massachusetts
    • 9Department of Pediatrics, Harvard Medical School, Boston, Massachusetts
    • 10Division of Sleep and Circadian Disorders, Department of Neurology, Brigham and Women’s Hospital, Boston, Massachusetts
    • 11Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, Ohio
    • 12Division of Hospital Medicine, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio
    • 13James M. Anderson Center for Health Systems Excellence, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio
    • 14Section of Pediatric Hospital Medicine, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania
    • 15Center for Pediatric Clinical Effectiveness, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania
    • 16Department of Pediatrics, Perelman School of Medicine at the University of Pennsylvania, Philadelphia
    • 17Pediatric Residency Program, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania
    • 18Division of Hospital Medicine, Children’s Hospital Los Angeles, Los Angeles, California
    • 19Department of Pediatrics, Keck School of Medicine, University of Southern California, Los Angeles
    • 20Division of Pediatrics, Children’s Hospital of Philadelphia Pediatric Care and Penn Medicine Princeton Medical Center, Philadelphia, Pennsylvania
    • 21Department of Pediatrics, University of Washington School of Medicine, Seattle
    • 22Center for Child Health, Behavior, and Development, Seattle Children’s Research Institute, Seattle, Washington
    • 23University of Washington Pediatric Residency Program, Department of Pediatrics, University of Washington, Seattle
    • 24Division of Hospital Medicine, Ann & Robert H. Lurie Children’s Hospital of Chicago, Chicago, Illinois
    • 25Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, Illinois
    • 26Department of Pediatrics, UT Southwestern Medical Center, Dallas, Texas
    • 27Division of Pediatric Hospital Medicine, Children’s Health Dallas, Texas
    • 28Division of General and Thoracic Surgery, Children’s Health Dallas, Dallas, Texas
    JAMA Netw Open. 2021;4(9):e2122826. doi:10.1001/jamanetworkopen.2021.22826
    Key Points

    Question  Are audit and feedback strategies and educational outreach associated with clinician perceptions of the feasibility, acceptability, appropriateness, and safety of continuous pulse oximetry use in children hospitalized with bronchiolitis who are not receiving supplemental oxygen (guideline-discordant use)?

    Findings  In this 6-hospital single-group nonrandomized clinical trial, 847 nurses and physicians highly rated the feasibility, acceptability, and appropriateness of audit and feedback strategies and educational outreach. Guideline-discordant pulse oximetry use decreased from 53% to 23% during the intervention period.

    Meaning  Educational outreach and audit and feedback strategies were feasible, acceptable, appropriate, and associated with a reduction in guideline-discordant continuous pulse oximetry use in children hospitalized with bronchiolitis who are not receiving supplemental oxygen.

    Abstract

    Importance  National guidelines recommend against continuous pulse oximetry use for hospitalized children with bronchiolitis who are not receiving supplemental oxygen, yet guideline-discordant use remains high.

    Objectives  To evaluate deimplementation outcomes of educational outreach and audit and feedback strategies aiming to reduce guideline-discordant continuous pulse oximetry use in children hospitalized with bronchiolitis who are not receiving supplemental oxygen.

    Design, Setting, and Participants  A nonrandomized clinical single-group deimplementation trial was conducted in 14 non–intensive care units in 5 freestanding children’s hospitals and 1 community hospital from December 1, 2019, through March 14, 2020, among 847 nurses and physicians caring for hospitalized children with bronchiolitis who were not receiving supplemental oxygen.

    Interventions  Educational outreach focused on communicating details of the existing guidelines and evidence. Audit and feedback strategies included 2 formats: (1) weekly aggregate data feedback to multidisciplinary teams with review of unit-level and hospital-level use of continuous pulse oximetry, and (2) real-time 1:1 feedback to clinicians when guideline-discordant continuous pulse oximetry use was discovered during in-person data audits.

    Main Outcomes and Measures  Clinician ratings of acceptability, appropriateness, feasibility, and perceived safety were assessed using a questionnaire. Guideline-discordant continuous pulse oximetry use in hospitalized children was measured using direct observation of a convenience sample of patients with bronchiolitis who were not receiving supplemental oxygen.

    Results  A total of 847 of 1193 eligible clinicians (695 women [82.1%]) responded to a Likert scale-based questionnaire (71% response rate). Most respondents rated the deimplementation strategies of education and audit and feedback as acceptable (education, 435 of 474 [92%]; audit and feedback, 615 of 664 [93%]), appropriate (education, 457 of 474 [96%]; audit and feedback, 622 of 664 [94%]), feasible (education, 424 of 474 [89%]; audit and feedback, 557 of 664 [84%]), and safe (803 of 847 [95%]). Sites collected 1051 audit observations (range, 47-403 per site) on 709 unique patient admissions (range, 31-251 per site) during a 3.5-month period of continuous pulse oximetry use in children with bronchiolitis not receiving supplemental oxygen, which were compared with 579 observations (range, 57-154 per site) from the same hospitals during the baseline 4-month period (prior season) to determine whether the strategies were associated with a reduction in use. Sites conducted 148 in-person educational outreach and aggregate data feedback sessions and provided real-time 1:1 feedback 171 of 236 times (72% of the time when guideline-discordant monitoring was identified). Adjusted for age, gestational age, time since weaning from supplemental oxygen, and other characteristics, guideline-discordant continuous pulse oximetry use decreased from 53% (95% CI, 49%-57%) to 23% (95% CI, 20%-25%) (P < .001) during the intervention period. There were no adverse events attributable to reduced monitoring.

    Conclusions and Relevance  In this nonrandomized clinical trial, educational outreach and audit and feedback deimplementation strategies for guideline-discordant continuous pulse oximetry use among hospitalized children with bronchiolitis who were not receiving supplemental oxygen were positively associated with clinician perceptions of feasibility, acceptability, appropriateness, and safety. Evaluating the sustainability of deimplementation beyond the intervention period is an essential next step.

    Trial Registration  ClinicalTrials.gov Identifier: NCT04178941

    Introduction

    Bronchiolitis is the leading reason for non–birth-related hospitalization in infants, accounting for more than 100 000 hospitalizations and $734 million in hospital costs annually.1 Bronchiolitis is a self-limited viral illness with a well-documented clinical course and national evidence-based practice guidelines.2-4 In addition to management recommendations, guidelines advise against continuous pulse oximetry monitoring (cSpo2) for patients who are not receiving supplemental oxygen, as it increases health care use without improving outcomes.2,3,5-9 Although none of the guidelines that discourage cSpo2 explicitly define it, we consider any use of pulse oximetry beyond a spot check (an in-person assessment of the oxygen saturation, with the probe applied and removed by staff during a single visit to the patient’s room) to represent continuous measurement. A recent study measured cSpo2 use at 56 US and Canadian hospitals and found that, overall, 46% of patients received cSpo2, suggesting a gap between the evidence-based guidelines and real-world clinical practice.10

    Implementation science seeks to overcome evidence-to-practice gaps.11,12 Audit and feedback and educational outreach are common implementation strategies that are effective in improving processes of care and clinical outcomes13 and are used frequently and successfully in quality improvement initiatives in pediatric hospital settings.14-17 However, the feasibility of audit and feedback and educational outreach and their associations with successful deimplementation in the pediatric hospital setting is limited mainly to quality improvement collaboratives, with diverse bundled interventions and multiple practices targeted for deimplementation.18,19

    In this study, our objective was to measure the feasibility, acceptability, appropriateness, and perceived safety of educational outreach and audit and feedback deimplementation strategies and their effect on cSpo2 use in hospitalized children with bronchiolitis who are not receiving supplemental oxygen.

    Methods

    This nonrandomized clinical trial was part of the Eliminating Monitor Overuse: SpO2 portfolio of projects, which previously measured baseline cSpo2 use in patients with bronchiolitis who were not receiving supplemental oxygen from December 1, 2018, through March 31, 2019, at 56 hospitals (trial protocol in Supplement 1).10 The intervention took place in 14 non–intensive care units in 5 freestanding children’s hospitals and 1 community hospital from December 1, 2019, through March 14, 2020. The institutional review board at Children’s Hospital of Philadelphia approved the study, and the remaining sites established reliance agreements with the reviewing institutional review board. All sites granted waivers of consent or parental permission per 45 CFR 46.116(f)(3), assent per 45 CFR 46.408(a), and Health Insurance Portability and Accountability Act authorization per 45 CFR 164.512(i)(2)(ii). For the staff questionnaires, sites granted waivers of consent documentation per 45 CFR 46.117(c)(1)(ii). This study followed the Transparent Reporting of Evaluations With Nonrandomized Designs (TREND) reporting guideline.

    Design

    This was a prospective, nonrandomized, single-group implementation feasibility trial with historical control data from the baseline period listed above. We invited hospitals that participated in the baseline study to express interest in pilot trial participation using an online form. Twenty-three hospitals expressed interest. In addition to the primary study site (Children’s Hospital of Philadelphia), we invited 5 sites with risk-standardized monitoring percentages of 60% or more during the baseline study aiming to optimize diversity of geography and community hospital participation, as well as availability of monitor data in the electronic health record for a related substudy.20

    The intervention consisted of 2 deimplementation strategies: educational outreach and audit and feedback (Figure 1). These strategies were chosen based on barriers and facilitators identified in prior qualitative research21 and were then mapped to deimplementation strategies during 2 stakeholder strategy development panels. The education and audit and feedback interventions were delivered in person to physicians, nurses, and respiratory therapists.

    Educational Outreach Intervention

    Educational outreach started before audit and feedback, then continued at a frequency determined by the site principal investigator based on study site needs. Education included 3 components: national guidelines for cSpo2 monitoring in patients with bronchiolitis,2-4 evidence supporting intermittent pulse oximetry rather than cSpo2 monitoring, and their hospital’s baseline cSpo2 use10 (eFigure 1 in Supplement 2). Site principal investigators tailored non–core content (eg, logos and location-specific context) and the setting of sessions. Sites were asked to target a 50% reduction in cSpo2 use.

    Audit and Feedback Intervention and cSpo2 Use Data Collection

    Study team members underwent webinar-based training in the fall of 2019. During the intervention period, local study teams conducted medical record review to screen patients for eligibility, followed by bedside data collection rounds using previously published methods10,22 to determine the cSpo2 status of patients with bronchiolitis not receiving supplemental oxygen. These data served as the outcome measure for cSpo2 use and the audit data for audit and feedback. Teams were encouraged to conduct data collection twice weekly, with timing based on availability of the data collector. Given that rapid improvements in clinical status are expected with bronchiolitis23 and that cSpo2 use should change accordingly, sites could collect data from the same patient on multiple occasions if observations were more than 6 hours apart. An identifier allowed accounting for clustering in analyses. Eligible patients were between the ages of 8 weeks and 23 months and admitted to a general medical service with a primary diagnosis of bronchiolitis. We excluded patients documented as premature or preterm and those with documented prematurity of less than 28 weeks’ gestation; cyanotic congenital heart disease or pulmonary hypertension; home oxygen use, positive pressure ventilation requirement, or tracheostomy; primary neuromuscular disease; immunodeficiency; or cancer. Additional exclusions were added in December 2019 (heart failure, myocarditis, or arrhythmia) and in March 2020 (COVID-19).

    The feedback intervention had 2 forms: individual real-time inquiry and weekly unit-level feedback. In real-time inquiry, if clinicians were available during cSpo2 audits, data collectors asked briefly and nonjudgmentally about the indication for cSpo2. In unit-level feedback, the study coordinating team summarized each participating unit’s data in a weekly unit-specific dashboard (an electronic document with unit-specific cSpo2 monitoring performance and reminders for specific practices to improve performance) sent to site principal investigators that reiterated educational outreach information (eFigure 2 in Supplement 2). In the dashboard, which site principal investigators tailored to site needs, data were compared with the performance of other hospital units, the hospital’s baseline performance,10 and the hospital’s target performance. The study coordinating team also suggested improvement targets based on local monitoring patterns, such as day and night variation.

    Outcomes

    The primary outcomes were acceptability, appropriateness, and feasibility of the deimplementation strategies.21 We also measured perceived safety of intermittently spot-checking oxygen saturation instead of using cSpo2. To estimate penetration24 of guideline-concordant care2-4 of patients with bronchiolitis not receiving supplemental oxygen, we assessed the change in cSpo2 use between the baseline10 and intervention periods.

    Implementation Outcomes

    Aiming to distribute a brief instrument with minimal overlap between questions, our multidisciplinary study team of experts in pediatrics, nursing, clinical research, patient safety, and implementation science parsimoniously selected and adapted items from the validated Acceptability of Intervention Measure, Intervention Appropriateness Measure, and Feasibility of Intervention Measure instruments25 for the study questionnaire. The team also developed questionnaire items focused on perceived safety, norms, and intentions using published guidance on constructing Theory of Planned Behavior–based questionnaires.26 The resulting questionnaire (eAppendix in Supplement 2) was distributed electronically to study unit clinicians, including nurses, advanced practice nurses, and resident, fellow, and attending physicians. Respiratory therapists were not included because their scope of practice does not include managing cSpo2 monitoring and their coverage is spread across multiple hospital units, making it difficult to identify individuals who worked on intervention units. Clinicians who reported caring for patients with bronchiolitis on intervention units but not being exposed to either intervention only completed the questions about perceived safety, norms, and intentions.

    Clinical Outcomes

    We report the percentage of patients with bronchiolitis not receiving supplemental oxygen who were receiving cSpo2 as “guideline-discordant monitoring” in the baseline and intervention periods. To report the estimated change in penetration24,27 of guideline-concordant care (avoiding cSpo2 for patients not receiving supplemental oxygen), we examined the reduction in cSpo2 between the baseline and intervention periods.

    Patient characteristics were abstracted from the electronic health record, including age, history of prematurity, sex, race, ethnicity, time since weaning from supplemental oxygen, history of apnea or a condition associated with neurologic impairment (eg, cerebral palsy), and presence of an enteral feeding tube.

    Adverse Event Surveillance

    We performed active surveillance for adverse events that could be associated with reductions in cSpo2. Staff at each site screened locally available data for code blue and rapid response team activations in any patients with bronchiolitis hospitalized on study units. Medical records of patients meeting these criteria were reviewed, and staff involved in the event were interviewed if necessary. If the patient was unmonitored during the event, there was further investigation. Events were considered at least possibly related to the study intervention if there was “a reasonable possibility that the adverse event may have been caused by the procedures involved in the research.”28(p1) Events determined to be at least possibly related to the study intervention were escalated according to local institutional review board protocols.

    Statistical Analysis

    For the questionnaire-based outcomes, we first summarized responses to each question descriptively. We then explored differences in responses between nurses and physicians using Pearson χ2 tests. We used ordinal logistic regression accounting for hospital-level clustering and reported odds ratios (ORs) with 95% CIs for nurses, with physicians as the reference group. The OR in ordinal logistic regression indicates the odds of choosing a response on the Likert scale 1 unit higher in agreement vs a response less than or equal to that level.29 Because these ORs can be difficult to interpret, for questions with significant differences in agreement between nurses and physicians, we calculated predictive marginal probabilities of each level of agreement29 and compared them by profession. We did not make any adjustments for multiple comparisons.30

    We calculated unadjusted guideline-discordant monitoring percentages for each hospital using the denominator of all directly observed patients with bronchiolitis who were not receiving supplemental oxygen and the numerator of patients who were simultaneously receiving cSpo2. We further compared baseline and intervention period data overall and at the hospital level using logistic regression, adjusted for the same covariates used in previous research, including age combined with preterm birth, time since weaning from supplemental oxygen, documented history of apnea or cyanosis during the present illness, presence of an enteral feeding tube, neurological impairment, and nighttime observation.10 To obtain adjusted estimates at the hospital level, we included an interaction term between hospital and the intervention period. The model accounted for clustering of observations within patient admissions.

    To report the estimated change in penetration24,27 of guideline-concordant care (avoiding cSpo2 for patients with bronchiolitis who were not receiving supplemental oxygen), we calculated the difference in the cSpo2 use percentage between the baseline and intervention periods, with the percentage point decrease in guideline-discordant monitoring corresponding to an equivalent percentage point increase in the penetration of guideline-concordant care.

    Prior to the study, we performed a power calculation based on the 5-point Likert-based Acceptability of Intervention Measure, Intervention Appropriateness Measure, and Feasibility of Intervention Measure questionnaire. Assuming an average feasibility response of “agree” (numerical value of 4.0) with an SD of 3 and alpha = 0.05, a sample size of 73 questionnaires per site would provide 80% power to reject the null hypothesis of a “neither agree nor disagree” response (numerical value of 3.0) at each site. We used data collection forms designed in Research Electronic Data Capture and hosted at Children’s Hospital of Philadelphia.31 We used Stata, version 16.0 (StataCorp LLC) for all analyses. All P values were from 2-sided tests and results were deemed statistically significant at P < .05.

    Results

    The intervention period took place in 14 units inside 6 hospitals and included 1051 audit observations (range, 47-403 per site) on 709 unique patient admissions (range, 31-251 per site) between December 1, 2019, and March 14, 2020. The initially planned study period extended to March 31, 2020, but closed early because of institutional restrictions imposed during the COVID-19 pandemic. During the intervention period, sites completed a median of 16 in-person education and/or data feedback sessions (range, 10-64 per site). Audit observations during the study period were compared with 579 observations (range, 57-154 per site) from the same hospitals during the baseline 4-month period (prior season) to determine if the strategies were associated with a reduction in use. Sites conducted 148 in-person educational outreach and aggregate data feedback sessions and provided real-time 1:1 feedback 171 of 236 times (72% of the time when guideline-discordant monitoring was identified).

    We distributed questionnaires to 1263 clinicians; 1193 clinicians were eligible and 847 responded, for an overall 71% response rate (range, 66%-84% among hospitals) (Figure 2). Of the 847 respondents, 474 reported attending at least 1 educational session, and 664 reported being provided with feedback data about their unit’s performance at least once; those respondents completed the corresponding questionnaire regarding acceptability, appropriateness, and feasibility. Additional details on response rates by profession are presented in eTable 1 in Supplement 2. Results are summarized in Table 1 and eTable 2 in Supplement 2.

    Educational Sessions

    Respondents rated educational sessions favorably. Most agreed or completely agreed that they liked (435 of 474 [92%]) and welcomed (455 of 474 [96%]) the intervention (acceptability), that it was appropriate (457 of 474 [96%]), and that it was feasible (424 of 474 [89%]) (eTable 2 in Supplement 2). There were no significant differences between physician and nurse responses about the educational sessions (Table 1).

    Audit and Feedback

    Respondents also rated the audit and feedback intervention favorably (eTable 2 in Supplement 2). Most agreed or completely agreed that they liked (615 of 664 [93%]) and welcomed (636 of 664 [96%]) the intervention (acceptability) and that the intervention was appropriate (622 of 664 [94%]) and feasible (557 of 664 [84%]). There were no significant differences between physician and nurse responses to questions about audit and feedback feasibility (Table 1). However, nurses had lower odds than physicians of agreeing that they welcomed continued feedback (a measure of acceptability; OR, 0.57; 95% CI, 0.33-1.00, P = .048).

    Acceptability, Feasibility, and Appropriateness

    With respect to measures of appropriateness, perceived safety, norms, and intentions, most respondents agreed or completely agreed that intermittently spot-checking oxygen saturation instead of cSpo2 in stable patients with bronchiolitis was safe (803 of 847 [95%]), was a good idea (783 of 847 [92%]) and helps reduce length of stay (776 of 847 [92%]). Only 4% of respondents (37 of 847) agreed that intermittently spot-checking oxygen saturation instead of cSpo2 put patients at risk, and only 15% (129 of 847) agreed that intermittently spot-checking oxygen saturation instead of cSpo2 was upsetting to parents. Compared with physicians, nurses had lower odds of agreeing that intermittently spot-checking oxygen saturation instead of cSpo2 monitoring is safe (OR, 0.28; 95% CI 0.24-0.33; P < .001) and that it is a good idea (OR, 0.28; 95% CI, 0.24-0.32; P < .001). These interprofessional contrasts were driven by differences in responses of “completely agree” vs “agree” (eTable 2 in Supplement 2). Nurses had significantly higher odds of agreeing that intermittently spot-checking oxygen saturation instead of cSpo2 is upsetting to parents (OR, 2.38, 95% CI, 1.87-3.02; P < .001) and puts patients at risk (OR, 3.91; 95% CI, 3.22-4.74; P < .001). Nurses had lower odds of reporting that their nurse colleagues prefer intermittently spot-checking oxygen saturation instead of using cSpo2 monitoring (OR, 0.40; 95% CI, 0.22-0.73; P < .001) and lower odds of agreeing that they intend to intermittently spot-check oxygen saturation instead of using cSpo2 in stable patients with uncomplicated bronchiolitis going forward (OR, 0.39; 95% CI, 0.34-0.46; P < .001).

    Clinical Monitoring Use

    Patient characteristics for the baseline and intervention periods are presented in Table 2. With respect to change in the practice of cSpo2, during the intervention period, 236 of 1051 patients (22%) with bronchiolitis who were not receiving supplemental oxygen received cSpo2 (hospital range, 4 of 47 [9%] to 110 of 255 [43%]) (Table 3), compared with 332 of 579 patients (57%) during the baseline period. Because of unit restructuring that occurred between the baseline and intervention periods (eg, changes in patient populations assigned to specific units and new units opening), directly comparing unit performance between the baseline and intervention periods was not possible. Using data from baseline period hospital units and adjusting for the same covariates used in the observational study’s analysis,10 guideline-discordant cSpo2 use decreased from 53% (95% CI, 49%-57%) to 23% (95% CI, 20%-25%; P < .001) during the intervention period (Table 3). The cSpo2 prevalence from the intervention period was 31 percentage points lower (95% CI, 26-35 percentage points) compared with baseline. This equates to a 31–percentage point increase in penetration24,27 of guideline-concordant care. There were no adverse events attributable to the intervention during the study.

    Discussion

    In this 6-hospital single-group cSpo2 deimplementation trial using historical control data, most respondents agreed that the deimplementation strategies targeting guideline-discordant cSpo2 for patients with bronchiolitis not receiving supplemental oxygen were acceptable, appropriate, feasible, and safe. Application of these strategies was temporally associated with a significant decrease in the adjusted percentage of hospitalized children with bronchiolitis not receiving supplemental oxygen who received guideline-discordant cSpo2.

    We noted important interprofessional differences in perceptions of intervention safety between nurses and physicians, which warrant further study and have implications for future deimplementation efforts. The deimplementation of practices considered safe in children may be particularly challenging.32 Continuous physiological monitoring has been widely adopted into clinical surveillance in various settings based on a common belief that it improves safety.33 Nurses in adult settings report that continuous physiological monitoring of patients outside of the intensive care unit improves patient safety.34 We found significant differences in the perceived safety of intermittently spot-checking oxygen saturation, with nurses rating this approach significantly lower than physician participants. The contrast was driven by differences in the distributions of “completely agree” vs “agree” responses, however, which may or may not be clinically important. This observed difference may relate to the scope of practice for nurses in pediatric hospital settings, where advocating for clinical decisions regarding the escalation or de-escalation of physiological monitoring to improve the detection of clinical deterioration is common.35 Furthermore, the education and audit and feedback interventions were delivered to each professional in the same format in this study. It is possible that tailoring the audit and feedback strategy to include clinicians setting role-specific goals, providing clinician-concordant benchmarks, and ensuring role concordance of the person delivering the feedback may improve perceptions of safety. Although clinicians overall rated the intervention favorably, the differences we observed in nurses’ perception of safety will need to be a focus of future deimplementation work in this area.

    Although our findings suggest that audit and feedback and education are associated with positive clinician perceptions and deimplementation of cSpo2, prior studies suggest that decay in improvement can occur after they are removed.36 A follow-up study of a multicenter learning collaborative found that many interventions were not sustained after the intervention period ended.37,38 Educational outreach strategies declined in use from 73% to 37% of hospitals, and data audits declined from 88% to 30%, with respondents citing insufficient time and competing priorities. These findings are consistent with the perspective that education-based interventions are a necessary component in multistrategy interventions but rarely result in sustained behavior changes alone.39 Education may require fewer resources to implement than audit and feedback, as suggested by the high feasibility ratings in our study. Future studies can elucidate mechanisms that contribute to the effectiveness of differing configurations, including the ideal “dose” of audit and feedback and educational outreach, and anticipate necessary adaptations depending on the setting.

    The specific aspects of the education and audit and feedback strategies’ association with cSpo2 deimplementation in children’s hospitals are not well-established. Three single-center quality improvement studies used audit and feedback or educational outreach alongside additional components, such as the use of champions, standard pathways, and order set modifications.14,16,17 A multicenter improvement study using education along with capacity building saw increases in intermittent pulse oximetry orders.15 Although these studies describe improvements in practice, most used multiple concurrent or sequential interventions with less focus on evaluating distinct strategies. Based on the generally favorable clinician perceptions, our findings support further testing of audit and feedback and educational outreach to address cSpo2 use for hospitalized patients with bronchiolitis. Additional interventions, such as champions and electronic health record–based interventions, should be assessed in future studies to determine whether they are associated with deimplementation.

    Limitations

    This study has several limitations. First, 5 of the 6 study hospitals were freestanding children’s hospitals, and all participated in the baseline measurement study10 in which some clinicians may have been exposed to education. Our findings may not be generalizable to all inpatient settings where patients with bronchiolitis receive care. Second, we used a convenience sampling approach for the monitoring practice audits, which may have resulted in a nonrepresentative sample. Third, our questionnaire response rate of 71% was adequate, but it remains possible that those with more negative experiences with the intervention chose not to participate, biasing our findings toward more positive views of deimplementation. Fourth, the historical controls were collected in the bronchiolitis season immediately preceding this study’s intervention period. It is possible that secular trends occurring in the months between bronchiolitis seasons may have influenced monitoring practices and clinician perceptions of the deimplementation intervention.

    Conclusions

    Educational outreach and audit and feedback deimplementation strategies were temporally associated with reductions in cSpo2 use during the intervention period. Clinicians rated these interventions favorably in terms of acceptability, appropriateness, feasibility, and perceived safety, suggesting that these are sensible strategies to begin addressing guideline-discordant cSpo2 use. Further studies should focus on how best to sustain improvements brought about by personnel-intensive interventions, such as educational outreach and audit and feedback, and seek to mechanistically understand key interprofessional, contextual, and strategy-specific characteristics associated with deimplementation success.

    Back to top
    Article Information

    Accepted for Publication: June 12, 2021.

    Published: September 2, 2021. doi:10.1001/jamanetworkopen.2021.22826

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Schondelmeyer AC et al. JAMA Network Open.

    Corresponding Author: Amanda P. Bettencourt, PhD, APRN, Department of Systems, Populations, and Leadership, University of Michigan School of Nursing, 400 N Ingalls St, Rm 4304, Ann Arbor, MI, 48109 (abetten@umich.edu).

    Author Contributions: Drs Xiao and Bonafide had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Drs Schondelmeyer and Bettencourt are co–first authors.

    Concept and design: Schondelmeyer, Bettencourt, Beidas, Wolk, Landrigan, Brady, Brent, Parthasarathy, Russell, Bonafide.

    Acquisition, analysis, or interpretation of data: Schondelmeyer, Bettencourt, Xiao, Kern-Goldberger, Sergay, Lee, Russell, Prasto, Zaman, McQuistion, Lucey, Solomon, Garcia, Bonafide.

    Drafting of the manuscript: Schondelmeyer, Bettencourt, Sergay, Garcia.

    Critical revision of the manuscript for important intellectual content: Schondelmeyer, Bettencourt, Xiao, Beidas, Wolk, Landrigan, Brady, Brent, Parthasarathy, Kern-Goldberger, Lee, Russell, Prasto, Zaman, McQuistion, Lucey, Solomon, Bonafide.

    Statistical analysis: Xiao, Sergay, Bonafide.

    Obtained funding: Bonafide.

    Administrative, technical, or material support: Wolk, Brent, Kern-Goldberger, Sergay, Garcia.

    Supervision: Schondelmeyer, Brady, Parthasarathy, Russell, Prasto, Bonafide.

    Conflict of Interest Disclosures: Dr Schondelmeyer reported receiving grant funding from the Agency for Healthcare Research and Quality (AHRQ) and the Association for the Advancement of Medical Instrumentation. Dr Bettencourt reported receiving funding from the National Heart, Lung, and Blood Institute (NHLBI). Dr Beidas reported receiving royalties from Oxford University Press; serving as a consultant to the Camden Coalition of Health Care Providers; serving on the Optum Behavioral Health Clinical and Scientific Advisory Committee; and receiving grant funding from the National Institute of Mental Health (NIMH), the National Cancer Institute, the National Institute for Nursing Research, NHLBI, the National Institute on Aging, the National Institute of Allergy and Infectious Diseases, the Centers for Disease Control and Prevention, the Patient-Centered Outcomes Research Institute, the Veterans Affairs Medical Center, the Substance Abuse and Mental Health Services Administration, and the National Psoriasis Foundation. Dr Wolk reported receiving grant funding from AHRQ, NIMH, NHLBI, the Health Resources and Services Administration, and the Institute of Education Sciences. Dr Bonafide reported receiving grant funding from the AHRQ and the National Science Foundation for research related to physiologic monitoring. No other disclosures were reported.

    Funding/Support: Research reported in this publication was supported by a Cooperative Agreement from the NHLBI of the National Institutes of Health (NIH) under award number U01HL143475 (Dr Bonafide). Dr. Schondelmeyer’s effort contributing to this manuscript was in part funded by the AHRQ under Award Number K08HS026763. Dr. Bettencourt’s effort contributing to this manuscript was in part funded by the NHLBI of the NIH under award number K12HL13803903.

    Role of the Funder/Sponsor: The funding organizations had no role in the design of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.

    Group Members: The members of the PRIS Network are listed in Supplement 3.

    Data Sharing Statement: See Supplement 4.

    Disclaimer: The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH or the AHRQ. As this was funded through an NIH Cooperative Agreement, NIH scientists participated in study conference calls and provided ongoing feedback on the conduct and findings of the study.

    Additional Contributions: The authors acknowledge the following individuals for their assistance with data collection and implementing interventions: Amera Al-Ali, BSN, RN, CPN, and Huong Mai, BSN, RN, CPN, Anne & Robert H. Lurie Children’s Hospital of Chicago; Waheeda Samady, MD, MSCI, and Kristin Van Genderen, MD, Anne & Robert H. Lurie Children’s Hospital of Chicago and Northwestern University; Gaylan Dascanio, MD, Yomna Farooqi, MD, and Sarah Khan, MD, Children’s Medical Center in Dallas, Texas, and UT Southwestern Medical Center; Caitlin Reaves, BSN, RN, CPN, and Hailee Scoggins, BSN, RN, MSN, Children’s Medical Center in Dallas, Texas; Laura Goldstein, MD, Jessica Hart, MD, and Muida Menon, RN, Children’s Hospital of Philadelphia; Joshua Frankland, MD, and Jennifer Moore, MD, Seattle Children’s Hospital; Amir Hassan, BA, Kyle Lamphier, MD, Maria Santos, MD, Sabrina Sedano, BS, and Susan Wu, MD, Children’s Hospital Los Angeles. None of these individuals were compensated beyond their normal salaries for their contributions to this study.

    References
    1.
    Fujiogi  M, Goto  T, Yasunaga  H,  et al.  Trends in bronchiolitis hospitalizations in the United States: 2000-2016.   Pediatrics. 2019;144(6):e20192614. doi:10.1542/peds.2019-2614 PubMedGoogle Scholar
    2.
    Ralston  SL, Lieberthal  AS, Meissner  HC,  et al; American Academy of Pediatrics.  Clinical practice guideline: the diagnosis, management, and prevention of bronchiolitis.   Pediatrics. 2014;134(5):e1474-e1502. doi:10.1542/peds.2014-2742 PubMedGoogle ScholarCrossref
    3.
    Quinonez  RA, Garber  MD, Schroeder  AR,  et al.  Choosing wisely in pediatric hospital medicine: five opportunities for improved healthcare value.   J Hosp Med. 2013;8(9):479-485. doi:10.1002/jhm.2064 PubMedGoogle ScholarCrossref
    4.
    Schondelmeyer  AC, Dewan  ML, Brady  PW,  et al.  Cardiorespiratory and pulse oximetry monitoring in hospitalized children: a Delphi process.   Pediatrics. 2020;146(2):e20193336. doi:10.1542/peds.2019-3336 PubMedGoogle Scholar
    5.
    Cunningham  S, Rodriguez  A, Adams  T,  et al; Bronchiolitis of Infancy Discharge Study (BIDS) group.  Oxygen saturation targets in infants with bronchiolitis (BIDS): a double-blind, randomised, equivalence trial.   Lancet. 2015;386(9998):1041-1048. doi:10.1016/S0140-6736(15)00163-4 PubMedGoogle ScholarCrossref
    6.
    McCulloh  R, Koster  M, Ralston  S,  et al.  Use of intermittent vs continuous pulse oximetry for nonhypoxemic infants and young children hospitalized for bronchiolitis: a randomized clinical trial.   JAMA Pediatr. 2015;169(10):898-904. doi:10.1001/jamapediatrics.2015.1746 PubMedGoogle ScholarCrossref
    7.
    Principi  T, Coates  AL, Parkin  PC, Stephens  D, DaSilva  Z, Schuh  S.  Effect of oxygen desaturations on subsequent medical visits in infants discharged from the emergency department with bronchiolitis.   JAMA Pediatr. 2016;170(6):602-608. doi:10.1001/jamapediatrics.2016.0114 PubMedGoogle ScholarCrossref
    8.
    Schuh  S, Freedman  S, Coates  A,  et al.  Effect of oximetry on hospitalization in bronchiolitis: a randomized clinical trial.   JAMA. 2014;312(7):712-718. doi:10.1001/jama.2014.8637 PubMedGoogle ScholarCrossref
    9.
    Schuh  S, Kwong  JC, Holder  L, Graves  E, Macdonald  EM, Finkelstein  Y. Predictors of critical care and mortality in bronchiolitis after emergency department discharge. J Pediatr. 2018;199:217-222.e1. doi:10.1016/j.jpeds.2018.04.010
    10.
    Bonafide  CP, Xiao  R, Brady  PW,  et al; Pediatric Research in Inpatient Settings (PRIS) Network.  Prevalence of continuous pulse oximetry monitoring in hospitalized children with bronchiolitis not requiring supplemental oxygen.   JAMA. 2020;323(15):1467-1477. doi:10.1001/jama.2020.2998 PubMedGoogle ScholarCrossref
    11.
    Eccles  MP, Mittman  BS.  Welcome to Implementation Science.   Implement Sci. 2006;1:1. doi:10.1186/1748-5908-1-1 Google ScholarCrossref
    12.
    Bauer  MS, Damschroder  L, Hagedorn  H, Smith  J, Kilbourne  AM.  An introduction to implementation science for the non-specialist.   BMC Psychol. 2015;3(1):32. doi:10.1186/s40359-015-0089-9 PubMedGoogle ScholarCrossref
    13.
    Jamtvedt  G, Young  JM, Kristoffersen  DT, O’Brien  MA, Oxman  AD.  Does telling people what they have been doing change what they do? a systematic review of the effects of audit and feedback.   Qual Saf Health Care. 2006;15(6):433-436. doi:10.1136/qshc.2006.018549 PubMedGoogle ScholarCrossref
    14.
    Schondelmeyer  AC, Simmons  JM, Statile  AM,  et al.  Using quality improvement to reduce continuous pulse oximetry use in children with wheezing.   Pediatrics. 2015;135(4):e1044-e1051. doi:10.1542/peds.2014-2295 PubMedGoogle ScholarCrossref
    15.
    Ralston  SL, Garber  MD, Rice-Conboy  E,  et al; Value in Inpatient Pediatrics Network Quality Collaborative for Improving Hospital Compliance with AAP Bronchiolitis Guideline (BQIP).  A multicenter collaborative to reduce unnecessary care in inpatient bronchiolitis.   Pediatrics. 2016;137(1). doi:10.1542/peds.2015-0851 PubMedGoogle Scholar
    16.
    Heneghan  M, Hart  J, Dewan  M,  et al.  No cause for alarm: decreasing inappropriate pulse oximetry use in bronchiolitis.   Hosp Pediatr. Published online January 30, 2018. doi:10.1542/hpeds.2017-0126 PubMedGoogle Scholar
    17.
    Mittal  V, Darnell  C, Walsh  B,  et al.  Inpatient bronchiolitis guideline implementation and resource utilization.   Pediatrics. 2014;133(3):e730-e737. doi:10.1542/peds.2013-2881 PubMedGoogle ScholarCrossref
    18.
    Ralston  SL, Garber  MD, Rice-Conboy  E,  et al. A multicenter collaborative to reduce unnecessary care in inpatient bronchiolitis. Pediatrics. 2016;137(1):e20150851. doi:10.1542/peds.2015-0851
    19.
    Kaiser  SV, Jennings  B, Rodean  J,  et al. Pathways for Improving Inpatient Pediatric Asthma Care (PIPA): a multicenter, national study. Pediatrics. 2016;145(6):e20193026. doi:10.1542/peds.2019-3026
    20.
    Rasooly  IR, Kern-Goldberger  AS, Xiao  R,  et al.  Physiologic monitor alarm burden and nurses’ subjective workload in a children’s hospital.   Hosp Pediatr. 2021;hpeds.2020-003509. doi:10.1542/hpeds.2020-003509PubMedGoogle Scholar
    21.
    Wolk  CB, Schondelmeyer  AC, Barg  FK,  et al; Pediatric Research in Inpatient Settings (PRIS) Network.  Barriers and facilitators to guideline-adherent pulse oximetry use in bronchiolitis.   J Hosp Med. 2021;16(1):23-30. doi:10.12788/jhm.3535 PubMedGoogle ScholarCrossref
    22.
    Rasooly  IR, Beidas  RS, Wolk  CB,  et al; Pediatric Research in Inpatient Settings (PRIS) Network.  Measuring overuse of continuous pulse oximetry in bronchiolitis and developing strategies for large-scale deimplementation: study protocol for a feasibility trial.   Pilot Feasibility Stud. 2019;5:68. doi:10.1186/s40814-019-0453-2 PubMedGoogle ScholarCrossref
    23.
    Mansbach  JM, Clark  S, Piedra  PA,  et al; MARC-30 Investigators.  Hospital course and discharge criteria for children hospitalized with bronchiolitis.   J Hosp Med. 2015;10(4):205-211. doi:10.1002/jhm.2318 PubMedGoogle ScholarCrossref
    24.
    Prusaczyk  B, Swindle  T, Curran  G.  Defining and conceptualizing outcomes for de-implementation: key distinctions from implementation outcomes.   Implement Sci Commun. 2020;1:43. doi:10.1186/s43058-020-00035-3 PubMedGoogle ScholarCrossref
    25.
    Weiner  BJ, Lewis  CC, Stanick  C,  et al.  Psychometric assessment of three newly developed implementation outcome measures.   Implement Sci. 2017;12(1):108. doi:10.1186/s13012-017-0635-3 PubMedGoogle ScholarCrossref
    26.
    Ajzen  I. Constructing a theory of planned behavior questionnaire. Published 2006. Accessed November 3, 2020. http://people.umass.edu/~aizen/pdf/tpb.measurement.pdf
    27.
    Proctor  E, Silmere  H, Raghavan  R,  et al.  Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda.   Adm Policy Ment Health. 2011;38(2):65-76. doi:10.1007/s10488-010-0319-7 PubMedGoogle ScholarCrossref
    28.
    Committee for the Protection of Human Subjects, The Children’s Hospital of Philadelphia. Unanticipated problems involving risks to subjects. Revised February 25, 2013. Accessed July 26, 2021. https://irb.research.chop.edu/sites/default/files/documents/irbsop408.pdf
    29.
    Mitchell  MN. Ordinal logistic regression. In:  Interpreting and Visualizing Regression Models Using Stata. Second ed. Stata Press; 2020.
    30.
    Rothman  KJ.  No adjustments are needed for multiple comparisons.   Epidemiology. 1990;1(1):43-46. doi:10.1097/00001648-199001000-00010 PubMedGoogle ScholarCrossref
    31.
    Harris  PA, Taylor  R, Thielke  R, Payne  J, Gonzalez  N, Conde  JG.  Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support.   J Biomed Inform. 2009;42(2):377-381. doi:10.1016/j.jbi.2008.08.010 PubMedGoogle ScholarCrossref
    32.
    Hoffman  JM, Keeling  NJ, Forrest  CB,  et al.  Priorities for pediatric patient safety research.   Pediatrics. 2019;143(2):e20180496. doi:10.1542/peds.2018-0496 PubMedGoogle Scholar
    33.
    Downey  CL, Chapman  S, Randell  R, Brown  JM, Jayne  DG.  The impact of continuous versus intermittent vital signs monitoring in hospitals: a systematic review and narrative synthesis.   Int J Nurs Stud. 2018;84:19-27. doi:10.1016/j.ijnurstu.2018.04.013 PubMedGoogle ScholarCrossref
    34.
    Watkins  T, Whisman  L, Booker  P.  Nursing assessment of continuous vital sign surveillance to improve patient safety on the medical/surgical unit.   J Clin Nurs. 2016;25(1-2):278-281. doi:10.1111/jocn.13102 PubMedGoogle ScholarCrossref
    35.
    Stotts  JR, Lyndon  A, Chan  GK, Bekmezian  A, Rehm  RS.  Nursing surveillance for deterioration in pediatric patients: an integrative review.   J Pediatr Nurs. 2020;50:59-74. doi:10.1016/j.pedn.2019.10.008 PubMedGoogle ScholarCrossref
    36.
    Gerber  JS, Prasad  PA, Fiks  AG,  et al.  Durability of benefits of an outpatient antimicrobial stewardship intervention after discontinuation of audit and feedback.   JAMA. 2014;312(23):2569-2570. doi:10.1001/jama.2014.14042 PubMedGoogle ScholarCrossref
    37.
    Biondi  EA, McCulloh  R, Staggs  VS,  et al; American Academy of Pediatrics’ REVISE Collaborative.  Reducing Variability in the Infant Sepsis Evaluation (REVISE): a national quality initiative.   Pediatrics. 2019;144(3):e20182201. doi:10.1542/peds.2018-2201 PubMedGoogle Scholar
    38.
    Kaiser  SV, Shadman  KA, Biondi  EA, McCulloh  RJ.  Feasible strategies for sustaining guideline adherence: cross-sectional analysis of a national collaborative.   Hosp Pediatr. 2019;9(11):903-908. doi:10.1542/hpeds.2019-0152 PubMedGoogle ScholarCrossref
    39.
    Soong  C, Shojania  KG.  Education as a low-value improvement intervention: often necessary but rarely sufficient.   BMJ Qual Saf. 2020;29(5):353-357. doi:10.1136/bmjqs-2019-010411 PubMedGoogle ScholarCrossref
    ×