aThree clinicians in the intervention group and 5 in the control group did not attend acute-care encounters during the study period.
The estimate of interest (and associated P value) is the treatment × time interaction term, representing the relative changes in trajectories before and during the intervention. Error bars indicate 95% CIs.
The estimate of interest (and associated P value) is the treatment × time interaction term, representing the relative changes in trajectories before and during the intervention. Error bars indicate 95% CIs. Y-axis in blue indicates range 0% to 30%.
Gerber JS, Prasad PA, Fiks AG, et al. Effect of an outpatient antimicrobial stewardship intervention on broad-spectrum antibiotic prescribing by primary care pediatricians: a randomized trial. JAMA. doi:10.1001/jama.2013.6287
eAppendix. Content Summaries of Clinician Education for Sinusitis, Pharyngitis, and Pneumonia
eFigure 1. Example Clinician Feedback Report
eFigure 2. Study Timeline
eTable 1. ICD-9 Codes and Laboratory Test Results Used for ARTI Case Definitions
eTable 2. ICD-9 Codes Used for Defining Non-ARTI Diagnoses Excluded From Analyses
eTable 3. Antibiotic Prescribing by Condition With Supplementary (Post Hoc) Pre-Post Analyses Confirming the Results of the Primary Analyses
Customize your JAMA Network experience by selecting one or more topics from the list below.
Gerber JS, Prasad PA, Fiks AG, et al. Effect of an Outpatient Antimicrobial Stewardship Intervention on Broad-Spectrum Antibiotic Prescribing by Primary Care Pediatricians: A Randomized Trial. JAMA. 2013;309(22):2345–2352. doi:10.1001/jama.2013.6287
Importance Antimicrobial stewardship programs have been effective for inpatients, often through prescribing audit and feedback. However, most antimicrobial use occurs in outpatients with acute respiratory tract infections (ARTIs).
Objective To evaluate the effect of an antimicrobial stewardship intervention on antibiotic prescribing for pediatric outpatients.
Design Cluster randomized trial of outpatient antimicrobial stewardship comparing prescribing between intervention and control practices using a common electronic health record. After excluding children with chronic medical conditions, antibiotic allergies, and prior antibiotic use, we estimated prescribing rates for targeted ARTIs standardized for age, sex, race, and insurance from 20 months before the intervention to 12 months afterward (October 2008–June 2011).
Setting and Participants A network of 25 pediatric primary care practices in Pennsylvania and New Jersey; 18 practices (162 clinicians) participated.
Interventions One 1-hour on-site clinician education session (June 2010) followed by 1 year of personalized, quarterly audit and feedback of prescribing for bacterial and viral ARTIs or usual practice.
Main Outcomes and Measures Rates of broad-spectrum (off-guideline) antibiotic prescribing for bacterial ARTIs and antibiotics for viral ARTIs for 1 year after the intervention.
Results Broad-spectrum antibiotic prescribing decreased from 26.8% to 14.3% (absolute difference, 12.5%) among intervention practices vs from 28.4% to 22.6% (absolute difference, 5.8%) in controls (difference of differences [DOD], 6.7%; P = .01 for differences in trajectories). Off-guideline prescribing for children with pneumonia decreased from 15.7% to 4.2% among intervention practices compared with 17.1% to 16.3% in controls (DOD, 10.7%; P < .001) and for acute sinusitis from 38.9% to 18.8% in intervention practices and from 40.0% to 33.9% in controls (DOD, 14.0%; P = .12). Off-guideline prescribing was uncommon at baseline and changed little for streptococcal pharyngitis (intervention, from 4.4% to 3.4%; control, from 5.6% to 3.5%; DOD, −1.1%; P = .82) and for viral infections (intervention, from 7.9% to 7.7%; control, from 6.4% to 4.5%; DOD, −1.7%; P = .93).
Conclusions and Relevance In this large pediatric primary care network, clinician education coupled with audit and feedback, compared with usual practice, improved adherence to prescribing guidelines for common bacterial ARTIs, and the intervention did not affect antibiotic prescribing for viral infections. Future studies should examine the drivers of these effects, as well as the generalizability, sustainability, and clinical outcomes of outpatient antimicrobial stewardship.
Trial Registration clinicaltrials.gov Identifier: NCT01806103
Antibiotics are the most common prescription drugs given to children.1 Although hospitalized children frequently receive antibiotics,2 the vast majority of antibiotic use occurs in the outpatient setting, roughly 75% of which is for acute respiratory tract infections (ARTIs).3
Unnecessary prescribing for viral ARTIs is well documented3-8 and has been declining.7-9 However, inappropriate prescribing also occurs for bacterial ARTIs, particularly when broad-spectrum antibiotics are used to treat infections for which narrow-spectrum antibiotics are indicated and recommended.1,3,6,8,9 The American Academy of Pediatrics (AAP) recommends penicillin or amoxicillin as first-line agents for streptococcal pharyngitis, acute sinusitis, and pneumonia10,11; however, roughly 50% of children receive broader-spectrum antibiotics for these common infections.3 Antimicrobial stewardship programs are recommended to optimize antimicrobial use in hospitalized patients,3-7,12-14 most often via prospective audit and feedback of antibiotic prescribing. In contrast, few recommendations for outpatients are offered, largely because of the paucity of data regarding effective interventions in ambulatory medicine.7,9,12,14
Because most antibiotic prescribing occurs among outpatients, we attempted to extend antimicrobial stewardship principles in that setting, focusing on the overuse of off-guideline broad-spectrum antibiotics for common ARTIs. Therefore, we designed a cluster randomized trial of clinician education coupled with audit and feedback of antibiotic prescribing across a large pediatric primary care network.
We conducted a cluster randomized trial evaluating an intervention to decrease inappropriate antibiotic prescribing for common ARTIs over time by primary care pediatricians. The intervention included clinician education coupled with audit and feedback of antibiotic prescribing to children with ARTIs. The primary outcomes were (1) change in broad-spectrum antibiotic prescribing for acute sinusitis, streptococcal pharyngitis, and pneumonia and (2) change in antibiotic prescribing for viral infections. We chose not to target otitis media, given concurrent implementation of a decision support tool for otitis media within a subgroup of practices.
Randomization. To ensure a similar distribution of practice characteristics between groups, we block-randomized practices (clusters) by location (urban, suburban, rural) and volume (encounters per year); 9 practices were randomized to the intervention and 9 to control. A representative from each practice, specifically chosen to speak on behalf of the group, consented to the study before treatment allocation and had no contact with the project statistician (A.R.L.), who performed the randomization, at any time. It was not possible for either the investigators or the primary care sites to be blinded to the intervention. Individual primary care practices were followed up over time, and all clinicians and patients in each enrolled practice were included in the intervention and control groups. The design was repeated cross-sectional because individual patients were not followed up over time and the observations used for analyses came from different sets of patients over time. Although the unit of observation was the clinician, we randomized at the practice level to avoid intrapractice contamination of the intervention.
Intervention. The intervention, spanning June 2010 through June 2011, included (1) a 1-hour clinician education session delivered on site by a physician member of the study team who is board-certified in pediatric infectious diseases (J.G.) to outline study goals, provide updates regarding current prescribing guidelines for common ARTIs (available as links to PDFs in the electronic health record; see eAppendix for education content summaries/guidelines), and present practice-specific baseline antibiotic prescribing data regarding these guidelines and (2) personalized audit and feedback of guideline-based antibiotic prescribing rates for the individual, the individual's practice, and the network of enrolled practices for viral infections, sinusitis, group A streptococcal pharyngitis, and pneumonia delivered every 4 months (see eFigure 1 for an example feedback report). Feedback reports were personalized, private, and delivered via secure office e-mail accounts and interoffice mail. A timeline of the intervention and data collection is outlined in eFigure 2. Clinicians from control practices did not receive the education or prescribing feedback; however, all practices were aware of their participation in a study during which prescribing patterns would be tracked. We included (and provided feedback to) clinicians who were practice members at the start of the intervention only.
The study was conducted within a hospital-affiliated network of 29 pediatric primary care sites. Because 2 clinician groups each served 3 distinct geographic locations (the remaining 23 practice groups served patients at 1 site only), there were effectively 25 distinct clinician practice groups (Figure 1). These included 5 primary academic (ie, student/resident training sites) and 20 community-based practices, most of which were formerly private groups that were acquired and integrated into the network. These practices served children of diverse racial and socioeconomic backgrounds within urban, suburban, and rural settings across southeastern Pennsylvania and southern New Jersey.
The Children's Hospital of Philadelphia Committee for the Protection of Human Subjects approved the study. Data were obtained from an electronic health record (EHR) (EpicCare, Epic Systems Inc), used exclusively by all practice sites for charting and medication prescribing for all office and telephone encounters. Patient-level data included age, sex, race, insurance type, and antibiotic allergies. Visit-level data included practice site, calendar month of encounter, encounter type (office, telephone, emergency department), purpose (preventive, nonpreventive), clinician type (physician, nurse practitioner, trainee), all International Classification of Diseases, Ninth Revision (ICD-9) codes associated with the encounter and on the “problem list,” streptococcal testing results (rapid and culture), and all prescriptions generated during the encounter.
Antibiotic receipt was defined as a prescription for an oral antibacterial agent associated with an office visit. Amoxicillin-clavulanate, second- and third-generation cephalosporins, and azithromycin (except for treatment of pneumonia) were considered broad-spectrum based on AAP prescribing guidelines for ARTIs; penicillin or amoxicillin are first-line recommended antibiotic therapy for acute sinusitis, streptococcal pharyngitis, and pneumonia.10,11 Case definitions for viral ARTIs included encounters with an ICD-9 code for a viral infection in the absence of an additional code for a bacterial infection or a positive group A streptococcal test result (eTable 1). Bacterial ARTIs required (1) an ICD-9 code for acute sinusitis, streptococcal pharyngitis, or pneumonia; (2) an associated antibiotic prescription (because the focus was on choice of antibiotics given the intention to treat a bacterial ARTI); and (3) for streptococcal pharyngitis, a positive rapid (uniformly available at point of care across practices) or culture-positive laboratory test result.
Case definitions were validated through iterative, manual chart review of fields collected electronically. We also examined free text (eg, physical examination, assessment/plan) in 100 randomly selected charts per bacterial diagnosis and in 200 charts for nonbacterial diagnoses. Data were collected from each practice from October 2008 through June 2011—20 months prior to and 12 months after the start of the intervention.
We developed statistical models for a repeated cross-sectional design in which clinics and clinicians were followed up over time, and we collected data on individual encounters. We grouped visits by month (from −20 to +12). Using a piecewise generalized linear model with a knot at month 0, the start of the intervention,15 we modeled the trajectory of the log odds of prescribing before and after the intervention between treatment and control practices. In this visit-level model, we standardized for patient-level covariates, including age category (<1, 1-5, 6-10, or >10 years), sex, race, and insurance type (Medicaid or not Medicaid). For all analyses, we excluded (1) preventive encounters, (2) ARTI encounters with an additional bacterial infection (eTable 2), (3) encounters with children with complex chronic conditions,16 and (4) encounters with children with allergy to antibiotics or children who received an antibiotic prescription within the prior 3 months. These prespecified exclusion criteria were chosen to best select a cohort of previously healthy children with new-onset ARTIs for which narrow-spectrum antibiotics were indicated.
Using this model, the between-group estimate of interest was the treatment × time interaction term, which represented the relative changes in trajectories of prescribing before and during the intervention between the 2 groups of practices. Comparison of trajectories of prescribing, rather than merely levels before and after the intervention, necessarily adjusted for the possibility that prescribing behavior was changing prior to and therefore without the benefit of the intervention. As these observations were necessarily clustered within primary care sites, and as both the levels and trajectories of prescribing varied across sites, all models calculated robust variances (and confidence intervals) using the sandwich estimator.15 Alternative approaches, most notably generalized linear mixed-effects models, would not converge to solutions.
For each analysis, we estimated the predictive margins17 to generate 66 (33 months × 2 treatments) standardized probabilities of prescribing for each combination of treatment group and month. These standardized probabilities used the same regression models that adjusted for possible differences in patient characteristics across primary care site and over time. To confirm results from the conventional statistical methods, we implemented bootstrap resampling to estimate 95% confidence bounds using 399 replicates by resampling practice sites with replacement within treatment strata.18 This resampling method preserves the correlation structure of the cluster randomized design. The 10th and 390th ordered values for standardized probabilities for each month and treatment group represented the middle 95% confidence bounds. Bootstrap-based P values confirmed those arising from robust variance estimates. Power calculations, performed at the cluster level with conservative estimates for the number of usable months prior to intervention and accounting for correlated observations within practices over time, suggested adequate power (>90%) to detect 10% point improvement in prescribing from the intervention; smaller differences in prescribing might be statistically significant but clinically unimportant.
Additionally, we report the results of an alternative analysis of model-based pre-post comparisons (without accounting for prescribing trajectories) along with raw numbers of prescriptions and visits over the 2 periods (eTable 3). Within-group changes of point estimates of antibiotic prescribing (time 0 and 12 months after intervention) between intervention and control groups were also computed. P values correspond to the primary analytic strategy accounting for the changes in trajectories within and between groups (outlined above).
All analyses were performed using Stata software, version 12.1 (Stata Corp), by means of the “logit” and “bootstrap” programs. Custom programs written in Stata produced the standardized probabilities according the definitions outlined by Korn and Graubard.17P values reflect 2-sided tests and P <.05 was considered statistically significant.
Of the network's 25 pediatric primary care practice groups, the 5 primary academic sites (ie, those dedicated to resident and student training) were excluded prior to randomization because (1) it would be infeasible to perform the intervention among the large and transient number of trainees and (2) baseline analyses revealed relatively low rates of off-guideline antibiotic prescribing patterns at these sites. Of the 20 eligible practice groups, 18 agreed to participate; 9 practice groups were randomized to the intervention group and 9 to the control group (Figure 1).
Overall, there were 1 291 824 office visits by 185 212 unique patients to 162 clinicians at 18 practices during the 32-month study period (October 2008 through June 2011). As shown in the Table, there was a good balance of practice location, patient volume, and number of practitioners between intervention and control practices. Practices randomized to the control group served more black children than those randomized to the intervention, but all results are standardized for patient characteristics.
Among children who were prescribed antibiotics for any indication, the overall proportion of antibiotic prescriptions that were broad-spectrum decreased from 26.8% to 14.3% (absolute difference, 12.5%) in the intervention group and from 28.4% to 22.6% (absolute difference, 5.8%) in control practices (difference of differences [DOD], 6.7%) during the 12 months following initiation of education/audit and feedback, excluding encounters with children with recent prior antibiotic use, chronic medical conditions, or antibiotic allergies and adjusting for age, sex, and insurance type (Figure 2). This difference between groups was significant when considering the relative changes in trajectories of broad-spectrum prescribing before and during the intervention between the 2 groups of practices (P = .01).
When stratifying by the individual bacterial ARTIs targeted by the intervention, broad-spectrum (off-guideline) antibiotic prescribing for pneumonia decreased from 15.7% to 4.2% in the intervention group and from 17.1% to 16.3% in the control group (DOD, 10.7%; P < .001 for trajectories) (Figure 3A). Broad-spectrum prescribing for acute sinusitis decreased from 38.9% to 18.8% in the intervention group and from 40.0% to 33.9% in the control practices (DOD, 14.0%; P = .12 for trajectories) (Figure 3B). Broad-spectrum prescribing for streptococcal pharyngitis started and remained low for both the intervention group (from 4.4% to 3.4%) and the control group (from 5.6% to 3.5%) (DOD, −1.1%; P = .82 for trajectories) (Figure 3C).
The baseline rate of any antibiotic prescribing for viral infections was low and did not change significantly after the intervention in either the intervention group (from 7.9% to 7.7%) or the control group (from 6.4% to 4.5%) (DOD, −1.7%; P = .93 for trajectories).
An alternative pre-post analysis confirmed these results using comparable models standardized for patient-level characteristics and adjusted for the cluster randomized design (eTable 3).
An outpatient antimicrobial stewardship intervention, consisting of clinician education coupled with personalized audit and feedback of antibiotic prescribing, significantly reduced off-guideline antibiotic use. This intervention nearly halved prescribing of broad-spectrum antibiotics to children during acute primary care encounters and decreased use of off-guideline antibiotics for children with pneumonia by 75% by 1 year after the intervention. Focusing on the overuse of broad-spectrum antibiotics for ARTIs for which narrow-spectrum agents are recommended allowed us to target high-frequency conditions with clear guidelines and adapt antimicrobial stewardship principles to the ambulatory setting.
This intervention focused on acute sinusitis, streptococcal pharyngitis, and pneumonia because (1) they are among the most frequent indications for antibiotic use in children; (2) clear, evidence-based, AAP-endorsed prescribing guidelines exist; and (3) previous studies have documented overprescribing of broad-spectrum antibiotics for these conditions.3,6,9,14 We chose not to target otitis media, given the concurrent implementation of a decision support tool for otitis media within a subgroup of practices. Although the clinician education sessions also discussed the ineffectiveness of antibiotic prescribing for viral infections, prescribing for viral infections was extremely low at baseline and did not change over the study period. The reasons for this are likely multifactorial but might be partially driven by the EHR association of antibiotic orders with encounter diagnosis codes—a disincentive to link antibiotic prescriptions to viral diagnoses—as well as the documented national trend in reduced prescribing for these conditions.7,9,14
A variety of strategies have attempted to improve antibiotic prescribing in primary care19-23 using generally time- and labor-intensive multipronged approaches. For example, a prior large, cluster randomized pediatric study bundled clinician education, guidelines, and limited prescribing feedback.20 This intervention differed with ours in its low clinician attendance at educational sessions, use of population-level data, and inability to audit and provide feedback of clinician-specific prescribing rates. Leveraging a comprehensive, shared electronic health record, our intervention supplied clinician-specific data including all encounters for the conditions targeted, generating personalized audit and feedback with peer benchmarking. Furthermore, the exclusion in the feedback reports of encounters by children with antibiotic allergies, chronic medical conditions, and prior antibiotic use—clinical scenarios that might warrant broad-spectrum antibiotic use—might have increased clinician confidence in the validity of the data.These details were communicated in education sessions attended by the vast majority of clinicians. This custom intervention is more akin to hospital-based antimicrobial stewardship programs, which often center on audit and feedback of clinician-specific prescribing based on specific guidelines.
Antimicrobial stewardship programs are recommended for hospitals and have been shown to reduce antimicrobial use, improve patient outcomes, and reduce health care costs for both adults and children.12,24-26 Our findings suggest that extending antimicrobial stewardship to the ambulatory setting, where such programs have generally not been implemented, may have important health benefits. Recently published guidelines endorsed by the Infectious Diseases Society of America, the Society for Healthcare Epidemiology of America, and the Pediatric Infectious Diseases Society acknowledge the importance of improving outpatient antibiotic use but do not offer specific recommendations to curb use in these settings because of the lack of data to guide such efforts.12 Because most antibiotic use occurs in outpatients, it is essential to apply stewardship principles to ambulatory medicine to maximize the population benefits of more judicious antibiotic use, including reduced antibiotic resistance pressure and unnecessary adverse drug effects and health care costs.
Our study had several limitations. First, the intervention was performed within a hospital-affiliated network of primary care pediatric practices using a shared electronic health record. Although this might limit generalizability, the practices enrolled served a diverse patient mix across urban, suburban, and rural settings, and the generic data elements used for audit and feedback should be common to all electronic health records—both for pediatric and adult patient care—which are now increasingly common in outpatient practices. Additionally, the automated feedback of our intervention required relatively few resources compared with more traditional hospital-based stewardship programs, which often rely on a team of clinical pharmacists and infectious diseases clinicians performing real-time interventions. Second, our data cannot isolate which element(s) (education, feedback, or simply visiting the practice) drove the reduction in prescribing. Furthermore, the trend in decreased broad-spectrum prescribing in control practices might have occurred due to contamination of the intervention across practice sites (ie, recognition of our auditing of antibiotic use). Third, because the cluster randomized study design lumps clinicians by practice (and was powered as such), we could not identify heterogeneity of treatment effect among individual clinicians. Fourth, we have not examined whether outcomes of infections were different between intervention and control groups; surveillance for those outcomes was beyond the scope of this intervention. Although it is possible that children receiving broad-spectrum antibiotics for ARTIs had better clinical outcomes than those receiving narrow-spectrum agents, this is unlikely given results of clinical trials that support current treatment guidelines. Fourth, we have not measured the sustainability of the effect, having observed antibiotic prescribing for only 12 months; it is unclear whether continued audit and feedback are necessary to sustain improvements in prescribing.
In conclusion, through clinician education coupled with audit and feedback of prescribing, we were able to significantly improve antibiotic use for children with bacterial ARTIs. This targeted application of antimicrobial stewardship principles to the ambulatory setting has the potential to affect the most common indications for antibiotic use. Future studies should examine the key drivers of these effects on antibiotic prescribing and the generalizability of findings to other health systems and measure the sustainability and clinical outcomes associated with differential prescribing patterns.
Corresponding Author: Jeffrey S. Gerber, MD, PhD, Division of Infectious Diseases, The Children's Hospital of Philadelphia, 3535 Market St, Ste 1518, Philadelphia, PA 19104 (firstname.lastname@example.org).
Author Contributions: Dr Gerber had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: All authors.
Acquisition of data: Gerber, Localio, Grundmeier, Wasserman, Zaoutis.
Analysis and interpretation of data: Gerber, Prasad, Fiks, Localio, Bell, Wasserman, Keren, Zaoutis.
Drafting of the manuscript: Gerber, Prasad, Localio.
Critical revision of the manuscript for important intellectual content: Prasad, Fiks, Localio, Grundmeier, Bell, Wasserman, Keren, Zaoutis.
Statistical analysis: Prasad, Fiks, Localio, Zaoutis.
Obtained funding: Gerber, Prasad, Localio, Grundmeier, Bell, Zaoutis.
Administrative, technical, or material support: Prasad, Fiks, Grundmeier, Bell.
Study supervision: Gerber, Fiks, Keren, Zaoutis.
Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Ms Prasad reports consultancy for the Taylor Collaboration. Dr Fiks reports consultancy for Nemours/PCORI and receipt of an honorarium from Elsevier. He is a coinventor of a clinical decision support software. Dr Keren reports board membership (2010-2012) for Tengion Inc and providing expert testimony for various entities. Dr Zaoutis reports consultancy for Merck, Pfizer, Astellas, Cubist, and Hemocue and payment for lectures/speakers bureaus from Merck. No other disclosures were reported.
Funding/Support: This research was supported by Agency for Healthcare Research and Quality contract HHSA2900200710013.
Role of the Sponsor: The funding agency had input regarding the design and conduct of the study and review and approval of the manuscript but had no input regarding the collection, management, analysis, or interpretation of the data, preparation of the manuscript, or decision to submit the manuscript for publication.
Online-Only Material: The Author Video Interview is available at here.
Additional Contributions: We thank the network of primary care clinicians and their patients and families for their contributions to clinical research through the Pediatric Research Consortium (PeRC) at The Children's Hospital of Philadelphia. In addition, we thank PeRC Director Jim Massey, RN, for his work on this project. Mr Massey received administrative support for this project, which included compensation from the funding agency for help with recruiting practices for the intervention.
Create a personal account or sign in to: