Effectiveness of Targeted Interventions on Treatment of Infants With Bronchiolitis: A Randomized Clinical Trial

Key Points

Question  Can the evidence-based treatment of infants with bronchiolitis be improved by using targeted interventions to deimplement low-value care?

Findings  In this international cluster randomized clinical trial of 26 hospitals and 3727 infants, an absolute risk difference favoring intervention hospitals was seen in compliance with 5 evidence-based recommendations in the treatment of infants with bronchiolitis.

Meaning  Use of targeted interventions improved the treatment of infants with bronchiolitis by deimplementing the use of ineffective and potentially harmful therapies and management; these results are important for bronchiolitis management, deimplementation science, and future interventions in acute care pediatrics.

Importance  In developed countries, bronchiolitis is the most common reason for infants to be admitted to the hospital, and all international bronchiolitis guidelines recommend supportive care; however, significant variation in practice continues with infants receiving non–evidence-based therapies. Deimplementation research aims to reduce the use of low-value care, and advancing science in this area is critical to delivering evidence-based care.

Objective  To determine the effectiveness of targeted interventions vs passive dissemination of an evidence-based bronchiolitis guideline in improving treatment of infants with bronchiolitis.

Design, Setting, and Participants  This international, multicenter cluster randomized clinical trial included 26 hospitals (clusters) in Australia and New Zealand providing tertiary or secondary pediatric care (13 randomized to intervention, 13 to control) during the 2017 bronchiolitis season. Data were collected on 8003 infants for the 3 bronchiolitis seasons (2014-2016) before the implementation period and 3727 infants for the implementation period (2017 bronchiolitis season, May 1-November 30). Data were analyzed from November 16, 2018, to December 9, 2020.

Interventions  Interventions were developed using theories of behavior change to target key factors that influence bronchiolitis management. These interventions included site-based clinical leads, stakeholder meetings, a train-the-trainer workshop, targeted educational delivery, other educational and promotional materials, and audit and feedback.

Main Outcomes and Measures  The primary outcome was compliance during the first 24 hours of care with no use of chest radiography, albuterol, glucocorticoids, antibiotics, and epinephrine, measured retrospectively from medical records of randomly selected infants with bronchiolitis who presented to the hospital. There were no patient-level exclusions.

Results  A total of 26 hospitals were randomized without dropouts. Analysis was by intention to treat. Baseline data collected on 8003 infants for 3 bronchiolitis seasons (2014-2016) before the implementation period were similar between intervention and control hospitals. Implementation period data were collected on 3727 infants, including 2328 boys (62%) and 1399 girls (38%), with a mean (SD) age of 6.0 (3.2) months. A total of 459 (12%) were Māori (New Zealand), and 295 (8%) were Aboriginal/Torres Strait Islander (Australia). Compliance with recommendations was 85.1% (95% CI, 82.6%-89.7%) in intervention hospitals vs 73.0% (95% CI, 65.3%-78.8%) in control hospitals (adjusted risk difference, 14.1%; 95% CI, 6.5%-21.7%; P < .001).

Conclusions and Relevance  Targeted interventions led to improved treatment of infants with bronchiolitis. This study has important implications for bronchiolitis management and the development of effective interventions to deimplement low-value care.

Trial Registration  Australian and New Zealand Clinical Trials Registry: ACTRN12616001567415.

Bronchiolitis is the most common respiratory condition affecting infants. In developed countries, bronchiolitis is the leading cause of admission to the hospital for infants,¹ with infants from indigenous and impoverished communities being most at risk.² This increased risk is, in part, the effect of structural racism resulting in indigenous populations being more likely to live in poverty and have reduced health services access, which leads to worse health outcomes.³ Bronchiolitis presentations occur at secondary as well as tertiary hospitals, and management occurs in both emergency departments (EDs) and pediatric wards; hence, evidence-based treatment needs to penetrate all levels of hospital and health systems.^4,5 Hospital admission costs alone are estimated in the US to exceed $1.7 billion per year.^1,6,7

Management of bronchiolitis is well defined internationally.¹ Guidelines recommend respiratory and hydration support,^8-12 and they recommend against the use of chest radiography (CR), albuterol, glucocorticoids, antibiotics, and epinephrine (eTable 1 in Supplement 1). Despite evidence that these 5 therapies and management processes are ineffective and associated with harm,^11-13 they continue to be widely used. In Australia and New Zealand (termed Australasia by the people of Australia and New Zealand), data from more than 3400 infant presentations to 7 hospitals demonstrated that at least 1 of these 5 interventions was used at least once in 27% to 48% of bronchiolitis admissions.¹⁴ These data are consistent with comparisons in North America, the United Kingdom, and Europe,¹⁵ highlighting the gap between the evidence-based and current clinical practices that exist internationally.

Implementation science aims to assess the effectiveness of interventions in translating research knowledge, reducing the gap between evidence-based practice and current clinical practice.¹⁶ Dissemination of a clinical practice guideline is seldom sufficient to change practice,¹⁷ with more active strategies being required to effect change. Strategies are more likely to be effective if underpinned by theories of behavior change and address both the barriers to and enhancers of recommended practice.^18,19 The Theoretical Domains Framework (TDF) incorporates a range of behavior change theories for use in implementation research.²⁰ The validated TDF²¹ has demonstrated strong explanatory and predictive powers across health care, including acute care settings, and is particularly useful when selecting interventions to improve practice.^22,23 Benefits of the TDF are that each domain has behavior change techniques linked to it, thereby optimizing use of techniques most likely to tackle identified issues,²⁴ with guidance available to assist in its use.²⁵ It was for all of these reasons that the TDF was used in our study. Less attention has focused on the challenge of reducing low-value health care and developing a theory and evidence to support deimplementation, which is of considerable importance for health care systems.^26,27

To our knowledge, no randomized clinical trials (RCTs) have been reported to determine the effectiveness of targeted interventions on bronchiolitis management.²⁸ Similarly, RCTs to deimplement unnecessary care in the acute care arena are rare. We undertook a cluster RCT to determine the effectiveness of targeted interventions vs passive dissemination of a clinical guideline to improve the evidence-based clinical care of infants presenting with bronchiolitis to the hospital. Minimizing harm caused by unnecessary interventions in the management of infants with bronchiolitis is an important patient- and family-centered outcome, and it is key to health care systems delivering evidence-based, cost-effective clinical management.

A detailed trial protocol has been published previously and is available in Supplement 2.²⁹ This study was a multicenter cluster RCT. Participating hospitals (clusters) were randomized between December 21, 2016, and February 3, 2017. The study was approved by the Royal Children’s Hospital Human Research Ethics Committee, Australia (HREC/16/RCHM/84), and the Northern A Health and Disability Ethics Committee, New Zealand (16/NTA/146). The ethics committee waived the requirement for informed patient consent because unidentified data from the medical records were being collected. This study followed the Consolidated Standards of Reporting Trials (CONSORT) reporting guideline.

This study included 26 hospitals (clusters) in Australia and New Zealand. Hospitals were eligible if they (1) had more than 135 bronchiolitis presentations per year, (2) were willing to be randomized to control or intervention, (3) had signed agreements from the ED and pediatric inpatient clinical directors, and (4) could retrospectively collect required data. Study design and recruitment are outlined in the Figure.

Following hospital consent, an independent statistician randomized hospitals using Stata software, version 14.2 (StataCorp LLC), stratifying by country and level of pediatric care (tertiary hospitals with dedicated intensive care units [ICUs] or secondary hospitals; randomization block size 2). Owing to the type of intervention, blinding was not possible.

Infants were eligible for inclusion if they were younger than 1 year at presentation and had both an ED and discharge diagnosis of bronchiolitis. Infant age younger than 1 year is consistent with bronchiolitis recommendations in Australia and New Zealand.¹¹ There were no other patient-level exclusion criteria.

Intervention hospitals received interventions targeting nursing and medical clinicians who managed infants with bronchiolitis in the ED and pediatric inpatient wards. Specific targeted interventions were developed using the TDF framework following a qualitative study³⁰ identifying local barriers and enablers to evidence-based bronchiolitis care. Table 1 details intervention components, eFigure 1 in Supplement 1 shows the stepped development approach, and the eAppendix in Supplement 1 details how interventions were rolled out according to the Template for Intervention Description and Replication checklist.³¹ The implementation period was the 2017 bronchiolitis season, May 1 to November 30, 2017.

Control hospitals received electronic and printed copies of the Australasian Bronchiolitis Guideline,¹¹ released December 2016; this was the first bronchiolitis guideline for use across Australia and New Zealand. Hospitals could undertake their usual dissemination activities. Information was not collected on how the guideline was disseminated owing to the possibility of the Hawthorne effect resulting from such a request.

Data collection occurred for the implementation period and the 3 preceding years, ensuring that the baseline data and effect of temporal trends could be assessed. All infants younger than 1 year presenting to each hospital with International Classification of Diseases, Ninth Revision (ICD-9) and International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) codes related to bronchiolitis were retrospectively identified (eTable 2 in Supplement 1). Research Electronic Data Capture (REDCap), version 8.5.1 (Vanderbilt University) was used to generate randomly selected presentations from identified lists for data extraction (n = 150 for implementation period; n = 100 for each of the previous 3 years). Auditors entered deidentified data extracted from medical records into the REDCap database, checking for eligibility. Reliability and accuracy was audited during site visits. Data included demographic details; ED and hospital or ICU length of stay; disposition; CR, albuterol, antibiotics (for respiratory cause), glucocorticoid, or any epinephrine administration during hospitalization; and death. Clinical leads at intervention hospitals maintained logs throughout the implementation period recording intervention use.

Analysis was by intention to treat. The primary outcome was the proportion of infants who complied with all 5 Australasian Bronchiolitis Guideline¹¹ recommendations known to have no benefit (CR, albuterol, glucocorticoids, antibiotics, and epinephrine) in the acute care period (first 24 hours of hospitalization). The acute care period was chosen as the primary outcome time frame because this management establishes the trajectory for further management in short-term admissions, such as bronchiolitis.

Secondary outcomes included (1) compliance with guideline recommendations (in ED, as inpatient, and during total hospitalization); (2) compliance with guideline recommendations (acute care period) for CR, albuterol, glucocorticoids, antibiotics, and epinephrine; (3) number of albuterol doses during the acute care period, during total hospitalization, while treated in the ED, and during inpatient treatment; (4) ICU admission; (5) length of hospital stay; and (6) death.

The primary outcome was analyzed at the individual patient level, adjusting for stratification factors at randomization (model A) and then adjusting for risk factors associated with bronchiolitis admission (model B). A priori subgroup analyses were planned if there was evidence of interaction between treatment group and subgroups: (1) infants with comorbidities vs without and (2) infants referred from another hospital or represented vs not. A priori sensitivity analyses were planned (1) at the cluster/hospital level, (2) excluding 2 hospitals without data on infants with ED length of stay less than 3 hours, and (3) using multiple imputation if primary outcome data were missing for more than 10% of the infants randomized for data extraction. Post hoc analyses were adjusted for preintervention compliance with all 5 bronchiolitis guideline recommendations (model A) to test any possible temporal trend in compliance in years preceding implementation affecting results and then adjusted for severity of illness (Australasian Triage Scale 1-2, immediately/imminently life-threatening vs 3-5, potentially life-threatening/potentially serious/less urgent).

Generalized linear models (GLMs) used the binomial family and an identity link function and calculated a cluster-robust SE to account for study site. Based on the GLM, risk differences (RDs) between guideline compliance proportions in the 2 groups and 95% CIs were computed. When GLM models did not converge, logistic regression with a cluster-robust standard error (hospital level) was used to calculate odds ratios.

Secondary outcomes were analyzed using GLM (model A). For outcomes not normally distributed, the GLM approach used the Poisson family and a log-link function and a cluster-robust SE (hospital level) to compute incidence rate ratios (IRRs) and 95% CIs.

Sample size calculation was based on the hypothesis that targeted interventions were superior to passive dissemination, resulting in an absolute increase in compliance with guideline recommendations of at least 15% in the intervention group compared with controls. This difference was clinically relevant. Previous data assumed baseline compliance of 52% to 73%.¹⁴ Using a conservative estimate of 50% compliance, 1620 infants per group in the implementation year were required to provide 82% power (α = .05), allowing for an average intraclass correlation coefficient (ICC) of 0.055¹⁴ (calculated from local data) and an average cluster size of 135 (n = 24 clusters; calculated in Stata 14.2). Allowing for possible attrition and resultant loss of power, 26 hospitals were recruited, each to report outcomes of 150 infants in the implementation year.

A statistical analysis plan (Supplement 3) was finalized before database lock. We present summary statistics as absolute and relative frequencies for categorical data and mean (SD) or median (interquartile range [IQR]) for continuous data. Differences between categorical variables are presented as RDs or odds ratios (95% CI). Differences between continuous data are presented as differences between mean (95% CI) or IRR (95% CI). All P values were 2-sided. Data were analyzed from November 16, 2018, to December 9, 2020.

Fifty-six hospitals were initially approached: 30 were unable to participate because they declined participation, had inadequate annual bronchiolitis presentations, or were unable to obtain timely governance or ethics approval. Twenty-six hospitals (Australian = 20; New Zealand = 6) were randomized (intervention group = 13; control group = 13), including 7 Australian tertiary pediatric hospitals (Figure). No hospitals withdrew after randomization.

Infants from intervention and control hospitals for the implementation year were well balanced in terms of baseline characteristics (Table 2), with compliance ranging from 64% to 73% and 60% to 66%, respectively, for the 3 years before the implementation year (n = 8003 infants) (eFigures 2 and 3 in Supplement 1). Overall, the 3727 implementation period participants had a mean (SD) age of 6.0 (3.2) months, with 2328 boys (62%) and 1399 girls (38%). A total of 459 (12%) identified as Māori (New Zealand) and 295 (8%) identified as Aboriginal/Torres Strait Islander (Australia); 505 (14%) had a history of prematurity (gestation <37 weeks), and 172 (5%) had comorbidities. Preadmission, at least 1 of the 5 key therapies and management processes known to have no benefit, was used in 653 infants (18%).

Compliance with the Australasian Bronchiolitis Guideline recommendations during the acute care period (first 24 hours of hospitalization) with no use of CR, albuterol, glucocorticoids, antibiotics, and epinephrine occurred in 1631 infants (85.1%; 95% CI, 82.6%-89.7%) in the intervention group and 1321 infants (73.0%; 95% CI, 65.3%-78.8%) in the control group (adjusted RD for baseline stratification factors, 14.1%; 95% CI, 6.5%-21.7%, P < .001; ICC, 0.11; 95% CI, 0.06-0.19). Adjusting for risk factors for bronchiolitis admission, sensitivity analyses at cluster level, and excluding 2 hospitals without data on infants with an ED length of stay less than 3 hours made no statistical difference to results. Sensitivity analysis for missing data was not undertaken as the 10% threshold was not reached (Table 3).

Compliance was improved in the intervention group for patients in the ED (RD, 10.8%; 95% CI, 4.1%-17.4%; P = .002), as inpatients (RD, 8.5%; 95% CI, 2.7%-14.3%; P = .004) and during the total hospitalization (RD, 14.4%; 95% CI, 6.2%-22.6%; P < .001) (Table 4). Absolute improvements in compliance occurred for each of the 5 guideline recommendations, with strong evidence for improvement in intervention patients in the use of albuterol (RD, 9.4%; 95% CI, 5.6%-13.2%; P < .001) and CR (RD, 6.2%; 95% CI, 0.5%-11.9%; P = .03) (Table 4).

No statistically significant difference between groups was observed for length of hospital stay (median intervention group, 12 [IQR, 2-42] hours; control group, 11 [IQR, 2-45] hours; IRR, 0.9; 95% CI, 0.7-1.2; P = .67) or ICU admission (intervention group, 63 [3%]; control group, 41 [2%]; RD, 0.4%; 95% CI, −0.2% to 1.0%; P = .21) (Table 4). There were no deaths in either group.

No statistically significant difference between groups was observed in the median number of medication doses during total hospitalization for infants who received any medications (intervention group median, 3 [IQR, 1-7]; control group median, 3 [IQR, 1-6] doses; IRR, 1.1; 95% CI, 0.7-1.6; P = .78). Similarly, no statistically significant difference between groups was observed in the median number of albuterol doses at any time point or from any managing team in infants who received any albuterol (Table 4).

All 13 intervention hospitals undertook the majority of the 6 intervention components as per the study protocol. Total intervention fidelity scores (site protocol compliance) varied from 55% to 98% (mean [SD], 78% [13%]). Twelve (92%) of the 13 hospitals identified a medical and nursing clinical lead for the study from both the ED and pediatric inpatient teams, with all remaining engaged for the duration of the implementation period. Forty-seven (90%) of the 52 clinical leads attended stakeholder meetings, and 42 (81%) of clinical leads attended the train-the-trainer workshop, with at least 1 from each hospital attending. Clinical leads were requested to train 80% of staff with the intervention PowerPoint presentation within the first month. Five hospitals (38%) achieved this objective, 5 hospitals (38%) trained more than 50% of staff, and 3 hospitals (23%) trained less than 50% of staff within the required time frame. All hospitals continued regular staff training throughout the implementation period. Other educational materials provided were used at all hospitals, ranging from 40% to 90%. Seven audit and feedback cycles (100%) were completed by all hospitals, with results disseminated to staff.

Our results show that targeted interventions designed to address factors influencing bronchiolitis management can improve the care delivered to infants with bronchiolitis and deimplement unnecessary care. Specifically, we found a 14.1% difference in rates of compliance during the first 24 hours of hospitalization, favoring the intervention group for all 5 bronchiolitis guideline recommendations, with the greatest change seen in albuterol and CR use. Improvement occurred for management within ED visits, as inpatients, and throughout total hospitalization.

There is increasing awareness of the problem of overuse of low-value care in health care.^27,32 Reducing the use of inappropriate health interventions is important for minimizing patient harm, maximizing resources, and improving evidence-based health care delivery.³³ Despite the fact that deimplementation is generally harder than implementing new evidence, few frameworks exist to guide deimplementation, and there is no “magic bullet.”²⁷ Our study is, to our knowledge, one of few using rigorous methods to show effectiveness in deimplementing low-value care and addressing the call to advance science in this field.^26,27 Although only conducted within Australia and New Zealand, the robust design means outcomes are likely applicable to other developed countries. The greatest change was seen in increased albuterol and CR compliance with guideline recommendations, where previously this compliance had been problematic.¹⁴ Although some clinical situations may warrant CR use in bronchiolitis, routine use is not warranted and is associated with increased inappropriate antibiotic use.¹⁴ This topic has been the subject of many international Choosing Wisely campaigns.³² Therefore, it is acceptable that compliance with CR is close to 100%, whereas compliance with not using the pharmacotherapies should be 100%. Inappropriate use of pharmacotherapies is 4 to 7 times higher in Europe and North America compared with Australia and New Zealand, and albuterol use is the largest aspect of care requiring deimplementation, which emphasizes the generalizability of our study.¹⁴ Hospitals with lower baseline compliance than ours may see a greater effect size, particularly if drivers of inappropriate care are similar.

Previous studies to improve bronchiolitis care have used a quality improvement design using before-and-after evaluation of multifaceted interventions,^34-38 with interventions developed by expert clinicians.^34,39,40 These studies may have shown decreased use of inappropriate investigations and pharmacotherapies, but reliability is weakened by their nonexperimental design, which can be subject to major confounders. Although interventions developed by expert clinicians may by chance address factors influencing bronchiolitis management, they are not developed specifically to address identified factors influencing the staff delivering care, as in our study.²⁹

Improving evidence-based bronchiolitis management and reducing exposure to low-value and potentially harmful therapies (important patient-centered outcomes for the infant and their family) was achieved without the unintended consequences of increased length of stay or ICU admissions. As length of stay is unchanged, cost savings will be limited to the cost (time and money) of the individual guideline recommendations that were deimplemented.

Our aim for 15% improvement after intervention was based on clinical importance, with power calculations assuming a conservative estimate of 50% compliance (local data showed 52% and 73% compliance).¹⁴ The final compliance difference of 14.1% improvement from a control group compliance of 73% was consistent with these assumptions and at the higher limit of improvements shown in implementation cluster RCTs.⁴¹ Although small secular trends of improved compliance occurred in the 3 years before the implementation year (eFigures 2 and 3 in Supplement 1), adjustment for these trends did not change the results (Table 3).

Intervention hospitals aimed to educate 80% of clinicians within the first month, as educating clinicians at the end of the bronchiolitis season would have had little effect on management. Only 38% of hospitals (n = 5) achieved this target, with a further 38% (n = 5) achieving greater than 50% education and education continuing throughout the intervention period in all hospitals. This outcome suggests that a lower target and continued education were effective. Whether compliance would have increased if all hospitals had achieved the original target is unknown. All hospitals completed the 7 audit and feedback cycles. Studies have demonstrated that showing clinicians their performance data compared with that of peers can be effective in improving guideline compliance.^42,43 Clinical leads may have potentially valued this intervention favorably, with real-time data seen as important to monitor compliance and target areas of noncompliance. Furthermore, anonymously benchmarking each hospital against the top hospital may have encouraged improved compliance and competition. Feedback strategies to staff were designed to be multifaceted, with clinical leads disseminating results as locally appropriate. Importantly, all hospitals continued using all interventions throughout the implementation period. Given the complex nature of the intervention, undertaken without additional clinical lead time or financial input, a mean overall intervention fidelity score of 78% from a pragmatic trial design represents robust intervention use and reinforces the acceptability of interventions. Feedback on interventions from clinical leads was positive.

Our study has potential limitations. Hospitals needed to have more than 135 bronchiolitis presentations per year to be eligible. Therefore, translating results to smaller or resource-limited hospitals needs to be considered cautiously. The study was only conducted within Australia and New Zealand. However, the 26 hospitals represented a diverse range of health care systems from the 2 countries and were comparable to the types of hospitals where children are managed across Australia and New Zealand.⁴⁴ Hospital heterogeneity, with varying involvement of ED and pediatric inpatient staff in bronchiolitis management, is closely representative of hospitals throughout the developed world, likely making results broadly generalizable. Data collection was retrospective and subject to information bias due to missing data. However, this bias is likely minimal because data concerning pharmacotherapies and CR are well recorded in medical records. Furthermore, prospectively collecting data at control hospitals was inappropriate because it would introduce risk of the Hawthorne effect. Patient eligibility required both an ED and final diagnosis of bronchiolitis for inclusion. This requirement was potentially subject to selection bias; eg, if a patient with true bronchiolitis underwent a CR that was misinterpreted as bronchopneumonia, the different discharge diagnosis would result in the case being ineligible for study inclusion. Thus, true compliance may be lower than reported. No prospective measurement was made on control hospital dissemination activities, unlike intervention hospital dissemination activities, again owing to the possibility of the Hawthorne effect. Finally, no hospital was 100% compliant with all intervention components. All activity occurred within local clinical leads’ nonclinical time and existing local educational programs. Despite these limitations, this pragmatic design is likely to make the improvements seen generalizable. Sustaining practice change is challenging within health care settings, and there are limited postimplementation studies describing this process. Sustainability of practice improvement beyond the intervention period is unknown, and further follow-up of hospitals is required.

The targeted interventions in this cluster RCT were effective in improving the treatment of infants with bronchiolitis without any identified negative consequences. Our deimplementation study followed a stepped process to identify who needed to do what differently. Our study first evaluated, then targeted, the drivers and behaviors of non–evidence-based practice, with behavior change techniques likely to change practice. We then evaluated these interventions in a cluster RCT. These results provide clinicians and hospitals with clear implementation strategies to address unnecessary treatment of infants with bronchiolitis.

Accepted for Publication: December 28, 2020.

Published Online: April 12, 2021. doi:10.1001/jamapediatrics.2021.0295

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Haskell L et al. JAMA Pediatrics.

Corresponding Author: Stuart R. Dalziel, PhD, The University of Auckland, Private Bag 92019, Auckland, New Zealand (s.dalziel@auckland.ac.nz).

Author Contributions: Dr Dalziel and Ms Orsini had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Drs Dalziel and Oakley were the principal investigators for New Zealand and Australia, respectively.

Concept and design: Haskell, Tavender, Wilson, O’Brien, Babl, Borland, Cotterell, Orsini, Johnson, Oakley, Dalziel.

Acquisition, analysis, or interpretation of data: Haskell, Tavender, Wilson, O’Brien, Babl, Schembri, Orsini, Sheridan, Johnson, Oakley, Dalziel.

Drafting of the manuscript: Haskell, Dalziel.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Haskell, Schembri, Orsini.

Obtained funding: Babl, Johnson, Oakley, Dalziel.

Administrative, technical, or material support: Haskell, Wilson, O’Brien, Johnson, Dalziel.

Supervision: Babl, Orsini, Sheridan, Oakley, Dalziel.

Other–collaborative effort of all authors to develop project proposal and see through the process of research implementation through regular meetings and email communication: Cotterell.

Conflict of Interest Disclosures: Ms Haskell reported receiving grants from the Health Research Council of New Zealand (19/140) during the conduct of the study. Dr Tavender reported receiving grants from the National Health and Medical Research Council during the conduct of the study. Ms Wilson reported receiving grants (no. 1058560) from the National Health and Medical Research Council, Australia, and nonfinancial support from the Victorian Government's Operational Infrastructure Support Program, Australia, during the conduct of the study. Ms Orsini reported receiving grants (no. 1058560) from the National Medical Research Council, Australia, and from the Victorian Government's Operational Infrastructure Support Program, Australia, during the conduct of the study. Dr Oakley reported receiving grants from the National Health and Medical Research Council, Australia, during the conduct of the study. Dr Dalziel reported receiving grants from the Health Research Council of New Zealand, grants from the National Medical Research Council, Australia, grants from the Cure Kids New Zealand Research support, and grants from the Starship Foundation during the conduct of the study. No other disclosures were reported.

Funding/Support: This study was supported in part by grant 1058560 from the National Health and Medical Research Council (Dr Oakley) and a grant from the Center of Research Excellence for Pediatric Emergency Medicine, Australia (Dr Oakley). This study was supported by the Victorian Government's Operational Infrastructure Support Program, Australia, and the Health Research Council of New Zealand (13/556). Ms Haskell’s time was partially funded by a Clinical Research Training Fellowship from the Health Research Council of New Zealand (19/140). Dr Dalziel’s time was partially supported by Cure Kids, New Zealand. Dr Babl’s time was partially funded by a grant from the Royal Children’s Hospital Foundation, Melbourne, Australia, and the National Health and Medical Research Council Practitioner Fellowship.

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

The PREDICT Network: Austin Hospital, Melbourne, Victoria, Australia: Celia Bagshaw, MBChB, FACEM, Catriona Fleming, MBBS (Hons), FRACP MPH, Emily Joules, BPharm, MPH, Jane Lamers, BN; Ballarat Base Hospital, Ballarat, Victoria, Australia: Andrew Crellin, MBBS, FACEM, Hoang Mai, BN, PGDip (Emerg), Rajesh Sannappareddy, MBBS, FACEM, FRCEM, David Tickell, MBBS, FACEM, Naomi White, BN, BParamedicine, PGCert (Paediatrics); Caboolture Hospital, Caboolture, Queensland, Australia: Constance Gray, MN, Lisa Kane, FRACP, MClinEd, Suzanne Kenny, MBBS, FACEM, Fiona Lirios, BN, PGCert (Midwifery and Emerg Nursing); Cairns Hospital, Cairns, Queensland, Australia: Lambros Halkidis, MBBS, MMED, FACEM, Jenae Ives, BNS, Janice Jeffery, RN, Amanda McConnell, MBChB, FACEM, MEmergHlth, Jennifer Magee, BN, PGDip (Emerg/Trauma), Katia Sciani, BN, PGCert (Emerg); Casey Hospital, Monash Health, Berwick, Victoria, Australia: Elizabeth Brown, Dip, Nursing and Midwifery, Samantha Fruzynsksi, MN (Paeds), Suzanne Harris, MEd, Emily Jagiello, BN, MHlth Service Mgt, Paul Machet, FRACP, Alastair Meyer, FACEM; Christchurch Hospital, Christchurch, New Zealand: Tracey Bruce, MN, Sarah Carr, MBChB, FACEM, Laura Joyce, MBChB, FACEM, Rebecca Lowery, BN, PGDip, HSc (Nursing), Thomas Townend, MBChB, FRACP, Antony Watson, BBioMedSc; Gold Coast University Hospital, Southport, Queensland, Australia: Joanna Cronin, RN, Rachael Dunning, MPH, Shane George, MBBS, FACEM, MPH, Riku Haataja, BN, Richele Hadfield, MN, Nicholas McIntyre, BA(Psych), BN, GCert Emerg; Hawkes Bay Hospital, Hastings, New Zealand: Mark Barlow, MBChB, FACEM, Sharon Payne, MN, Melissa Spooner, DipN, Kai Steinmann, FRACP MD; Ipswich Hospital, Ipswich, Queensland, Australia: Corey Cassidy, MBBS (Hons), FACEM, Daryl Jensen, MAdvNP, Justin Jin, MBBS, Ruth McCaffery, RN, Bheemsain Rajpal, DCH, FRACP; Logan Hospital, Logan, Queensland, Australia: Brooke Charters, BN, PGCert (Paeds), Jan Cullen, MBBS, FRACP, Chelsea Johnstone, MBBS, Smriti Joshi, MBBS, DCH, Ben Lawton, FRACP, MPH, Stephen Margolis, MBBS, MD, FACRRM, Kerri Winstanley, FACEM; Lyell McEwin Hospital, Adelaide, South Australia, Australia: Aman Anand, FACEM, Sonya Kloeden, BN, GDip, NSc (Emer), Mark Thesinger, FRACP; Queensland Children’s Hospital, Brisbane, Queensland, Australia: Kate Cogill, BN, Nikola Creasey, MBChB, Kristy Kimlin, MBBS, FRACP, Matthew Lynch, MBBS, Janarthan Muraliharan, MBBS, Kate Nolan, MN, Natalie Phillips, MBBS, MPhil, Eadon Seifried, BN, MHM, Gabrielle Van Andel, BN, Joseph Walker, MD; Rotorua Hospital, Rotorua, New Zealand: Louise Couch, MBChB, FACEM, Sonja Crone, MBChB, FRACP, Dana Donaldson, BN, Janine Fowler, DipN, PGCert HSc, Janet France-Goss, BN, PGCert HSc, Kaidee Hesford, BN, Sally Hutton, BN, PGCert HSc; Royal Darwin Hospital, Darwin, Northern Territory, Australia: Anna Lithgow, FRACP, Klytie McKittrick, FRACP, Gina Majid, BScN, MHlth Service Mgt, Clare Mitchell, MBChB, FACEM, Peter Morris, PhD, FRACP; Royal North Shore Hospital, Sydney, New South Wales, Australia: Hannah Grieve, BN, Richard Lennon, MBBS, FRACP, FACEM, MBioeth, Lauren Shumack, BN, MPH, Joshua Steadson, MBBS (Hons), FRACP, Sarah Thompson, BN, Martin Ward, RN, Carola Wittekind, MBBS, FRACP, MHA; Southland Hospital, Invercargill, New Zealand: Jenny Cross, MBChB, FACEM, Sarah Halder, BN, PGDip, HSc, Marie Irvine, DipN, PGCert (Leadership and Management), Olivia Murray, MN, Ian Shaw, MBChB, FRACP, Annie Smith, MN, Amber Watkinson, BN, Martin Watts, MBChB, FACEM; Sydney Children’s Hospital, Sydney, New South Wales, Australia: Elizabeth Argent, MBBS, FRACP, Julie Dihm, BN, GDip, ClinPract, Coralie Gower, MClinPract (Paeds), Claire Jardine, MBChB, FACEM, Yvonne Mullins, MN, Arjun Rao, MBBS, MAppSci, FRACP, Shalika Shetty, MBBS, FRACP, Sophie Watkins, BN, GCert (Paed Emerg); Tauranga Hospital, Tauranga, New Zealand: Robyn Attwood, RN, PGDip, HlthSc (Nursing), Karyne Coker, BN, Jo Cole, MBChB, FACEM, PEM, Brooke Gordon, BN, Tambra Trist, MBChB, FRACP; The Children’s Hospital at Westmead, Sydney, New South Wales, Australia: Jacqueline Dalby-Page, PhD, Kathryn Green, MN, Megan Jones, BN, Mary McCaskill, MBBS, FACEM, Deepali Thosar, MBBS, DCH; The Northern Hospital, Melbourne, Victoria, Australia: Czarina Calderon, FRACP, Yiannis Efstathiadis, MD, Janice Fernandes, MN, Courtney Lowry, BN, Megan Robb, MBChB, FACEM; The Prince Charles Hospital, Brisbane, Queensland, Australia: Rose Fahy, FRACP, Samantha Giess, BN, Sarah Hazelwood, MHlthSc, Frances Kinnear, FACEM, PhD, Tanya Mountford, RN, David Wood, MBChB, FRACP, PEM, FAFOEM; Toowoomba Base Hospital, Toowoomba, Queensland, Australia: Jill Codd, RN, Alex King, MBChB, FACEM, Stacey Smith, BN, GCert (Child and Youth); Townsville Hospital, Townsville, Queensland, Australia: Donna Budge, MN, Nicole Chapman, MEmergN, Susan Montgomery, BN, Julia Peters, MBBS, FACEM, Ramaa Puvvadi, MBBS, MD, FRACP, PhD; Waikato Hospital, Hamilton, New Zealand: Christine Brabyn, MBChB, FRACP, FACEM, Kirsty Greaves, BN, PGDip, Nicky Nelson, BN, PGDip, Alexandra Wallace, MBChB, FRACP, PhD; Wollongong Hospital, Wollongong, New South Wales, Australia: Simon Binks, FACEM, FRCEM, MRCPCH, BM, BS, BMedSci, Tracey Couttie, BN, GCert (Paeds Critical Care), Tegan Zanotti, BN, GDip (Paeds), Joanne Morris, MBBS, FRACP, Susan Piper, MBBS, FRACP, Richard Pryor, MD, FACEM; Women and Children’s Hospital, Adelaide, South Australia, Australia: Christina Glowik, RN, Amit Kochar, MBBS, FRACP, MPH, MD, Gaby Nieva, GDip (Critical Care Nursing), David Thomas, MD.

Data Sharing Statement: See Supplement 4.

Additional Contributions: We thank all the hospitals and staff involved in our study. We thank Luke Stevens, BSc, for managing REDCap data coordination, and Marian Chandler for assistance with study organization and coordination of the train-the-trainer workshops, both from Murdoch Children’s Research Institute, Melbourne, Australia.