Figure 1. Printed decision support tool. This 11 × 17-inch poster was laminated and posted in each examination room of practices assigned to the printed decision support intervention arm during the intervention period. The algorithm is based on an evidence-informed approach to assessing pneumonia in adults with acute cough illness.15
Figure 2. Determination of uncomplicated acute bronchitis visits in the final study population. *Comorbidities include chronic lung disease, congestive heart failure, human immunodeficiency virus, cystic fibrosis, and malignant neoplasm. †Antibiotic-responsive secondary diagnoses include sinusitis, pharyngitis, otitis media, and pneumonia. CDS indicates computer-assisted decision support intervention; PDS, printed decision support intervention.
Figure 3. Effect of decision support strategies on antibiotic prescription rates for adolescents and adults diagnosed as having uncomplicated acute bronchitis. Error bars for each estimate reflect 95% CIs.
Figure 4. Distribution of changes in clinician-level antibiotic prescription rates for adolescents and adults diagnosed as having uncomplicated acute bronchitis following implementation of the decision support interventions. Each bar represents the absolute difference in antibiotic prescription rates between intervention and baseline years for an individual clinician. Clinicians were required to have at least 10 visits in the baseline and intervention periods to be eligible for this analysis. Statistical comparison between groups yielded the following P values: P = .006 for control sites vs printed decision support intervention sites, P = .003 for control sites vs computer-assisted decision support intervention sites, and P = .84 for printed decision support intervention sites vs computer-assisted decision support intervention sites.
Author Ralph Gonzales, MD, MSPH discusses A Cluster Randomized Trial of Decision Support Strategies for Reducing Antibiotic Use in Acute Bronchitis
Customize your JAMA Network experience by selecting one or more topics from the list below.
Gonzales R, Anderer T, McCulloch CE, et al. A Cluster Randomized Trial of Decision Support Strategies for Reducing Antibiotic Use in Acute Bronchitis. JAMA Intern Med. 2013;173(4):267–273. doi:10.1001/jamainternmed.2013.1589
Author Affiliations: Division of General Internal Medicine, Department of Medicine, and Department of Epidemiology and Biostatistics, University of California, San Francisco (Drs Gonzales and McCulloch and Ms Maselli); and Susquehanna Health, Williamsport (Dr Anderer), Geisinger Health System, Danville (Drs Bloom and Graf and Mss Stahl, Yefko, and Molecavage), and Division of General Internal Medicine, Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia (Dr Metlay).
Background National quality indicators show little change in the overuse of antibiotics for uncomplicated acute bronchitis. We compared the effect of 2 decision support strategies on antibiotic treatment of uncomplicated acute bronchitis.
Methods We conducted a 3-arm cluster randomized trial among 33 primary care practices belonging to an integrated health care system in central Pennsylvania. The printed decision support intervention sites (11 practices) received decision support for acute cough illness through a print-based strategy, the computer-assisted decision support intervention sites (11 practices) received decision support through an electronic medical record–based strategy, and the control sites (11 practices) served as a control arm. Both intervention sites also received clinician education and feedback on prescribing practices, as well as patient education brochures at check-in. Antibiotic prescription rates for uncomplicated acute bronchitis in the winter period (October 1, 2009, through March 31, 2010) following introduction of the intervention were compared with the previous 3 winter periods in an intent-to-treat analysis.
Results Compared with the baseline period, the percentage of adolescents and adults prescribed antibiotics during the intervention period decreased at the printed decision support intervention sites (from 80.0% to 68.3%) and at the computer-assisted decision support intervention sites (from 74.0% to 60.7%) but increased slightly at the control sites (from 72.5% to 74.3%). After controlling for patient and clinician characteristics, as well as clustering of observations by clinician and practice site, the differences for the intervention sites were statistically significant from the control sites (P = .003 for control sites vs printed decision support intervention sites and P = .01 for control sites vs computer-assisted decision support intervention sites) but not between themselves (P = .67 for printed decision support intervention sites vs computer-assisted decision support intervention sites). Changes in total visits, 30-day return visit rates, and proportion diagnosed as having uncomplicated acute bronchitis were similar among the study sites.
Conclusions Implementation of a decision support strategy for acute bronchitis can help reduce the overuse of antibiotics in primary care settings. The effect of printed vs computer-assisted decision support strategies for providing decision support was equivalent.
Trial Registration clinicaltrials.gov Identifier: NCT00981994
The overuse of antibiotics for acute respiratory tract infections (ARIs) is an important contributor to worsening trends in antibiotic-resistance patterns among community-acquired pathogens. In the United States among persons 5 years and older, ARIs in 2006 accounted for 8% of all visits to ambulatory practices and emergency departments and for 58% of all antibiotics prescribed in these settings.1 Particularly relevant to reducing total antibiotic use are bronchitis, the common cold, and nonspecific upper respiratory tract infections because most of these illnesses have a viral origin and do not benefit from antibiotic treatment.2,3 About 30% of office visits for the common cold and for nonspecific upper respiratory tract infections, as well as up to 80% of all visits for bronchitis, are treated with antibiotics in the United States each year.4-7 Although antibiotic prescribing for ARIs among children has declined and is less than that among adults, antibiotic prescribing for acute bronchitis (when this diagnosis is used among children) has not changed.1,4-7
Although national and local efforts seem to have helped reduce antibiotic use for some ARIs, reducing antibiotic treatment of acute bronchitis remains a challenge. Combining patient and physician education and feedback has been shown to help decrease antibiotic treatment of uncomplicated acute bronchitis in various environments, such as outpatient practices, urgent care clinics, and emergency departments.8-10 However, on a national level, antibiotic prescribing for uncomplicated acute bronchitis not only is not declining like it is for otitis media and for nonspecific upper respiratory tract infections but also seems to be worsening. The National Committee for Quality Assurance's Healthcare Effectiveness Data and Information Set includes a measure of the mean percentage of adult visits for uncomplicated acute bronchitis with antibiotic treatment; among participating health plans, the measures were 71.3%, 74.6%, 75.4%, and 76.0% for 2006 through 2009, respectively.11
One feature that makes acute bronchitis evaluation and treatment unique from other ARIs is the potential for the clinician to miss the diagnosis of pneumonia, a common and potentially life-threatening condition in the differential diagnosis of acute cough. In the emergency department setting, a substantial decrease was observed in antibiotic treatment of uncomplicated acute bronchitis (from 51% to 31%) when clinicians were provided with a simple clinical algorithm for estimating the probability of pneumonia among patients with acute cough illness.12 Extending and adapting this approach to outpatient practices was the goal of the present study.
In the outpatient setting, we considered 2 different options for implementing the simple clinical algorithm at the point of care, namely, traditional printed decision support (PDS) (often using an algorithmic approach) vs a computer-assisted decision support (CDS) strategy integrated into the work flow of an electronic health record (EHR). Although great enthusiasm abounds for the potential of EHRs to support guideline implementation efforts, investigators and clinicians have also reported that CDS tools can be resource intensive.13,14 This study describes the effect of PDS and CDS strategies compared with a control group in introducing a clinical algorithm for acute cough illness at the point of care among primary care practices with a mature EHR system.
Geisinger Health System is a nonprofit physician-led, fully integrated health care delivery system in rural central and northeast Pennsylvania with a service area population base of 2.6 million persons in 43 counties. The health system in 2010 consisted of 2 medical centers, an 829-physician multispecialty group, and a 247 000-member health plan. The Community Practice Service Line of approximately 200 primary care clinicians located in 37 practices across the service area had completed more than 894 000 outpatient visits in 2010. Geisinger Health System has been using a fully integrated EHR introduced in ambulatory care since 1996.
We conducted a 3-arm cluster randomized trial of different implementation strategies to reduce antibiotic use for uncomplicated acute bronchitis, including a PDS strategy arm, a CDS strategy arm, and a control arm. We excluded 4 practices because the annual number of visits with a primary diagnosis of bronchitis (International Classification of Diseases, Ninth Revision, Clinical Modification [ICD-9-CM ] code 466 or 490) was less than 100 in a sample of 3 recent years. One practice served as a test site for the CDS development and was automatically assigned as a control site. Among the remaining practices, 9 large practices (with 9000-15 000 annual patient visits) were randomly assigned to each study arm, and among the remaining 23 smaller practices (with 2000-9000 annual patient visits), we randomly assigned 8 practices to the PDS and CDS arms each and 7 practices to the control arm.
Experience has shown that multidimensional implementation strategies are often more successful than strategies using a single intervention tool.15 The PDS and CDS arms both received intervention components guided by the PRECEDE-PROCEED (predisposing, reinforcing, and enabling constructs in educational diagnosis and evaluation–policy, regulatory, and organizational constructs in educational and environmental development) model, which included tools addressing predisposing factors (clinician education and practice guidelines), reinforcing factors (clinical champions and audit and feedback), and enabling factors (patient education and decision support).16 Clinician education was delivered for each clinic by a clinical champion who participated in a half-day training session (led by R.G. and J.P.M.). Clinical champions were provided with data about their specific clinic's performance on the acute bronchitis Healthcare Effectiveness Data and Information Set measure and with a teaching slide set to use when reviewing this information with the clinicians in their respective clinics. Patient education was provided through brochures published by the Centers for Disease Control and Prevention.17
At PDS intervention sites, patient educational brochures were provided by triage nurses to patients with cough illness as part of routine care, and a poster displaying the clinical algorithm decision support was placed in all examination rooms18 (Figure 1). At CDS intervention sites, when triage nurses entered cough as the chief symptom in the EHR, a best practice electronic alert for the nurse would appear, prompting the nurse to provide the educational brochure to the patient to read before being evaluated by the clinician. At CDS intervention sites, the algorithm was programmed into the health system's EHR (EpicCare; Epic Systems Corporation) by the health system's information technology staff, with input by physicians and nurses from a clinic site assigned to the control group; the goals for the tool were to enhance work efficiency, adapt to complex visits, and provide high-level documentation. The key features included a structured template for documenting relevant history and physical examination elements in patients with ARIs broadly. These elements provided the data necessary to categorize a patient's probability of having pneumonia based on the acute cough clinical algorithm. Groups of electronic order sets (SmartSets; Epic Systems Corporation) were created that simplified relevant testing and treatment options for bronchitis, pneumonia, sinusitis, influenza, and nonspecific upper respiratory tract infections.
Patients who were targeted by the intervention included all adolescents and adults (≥13 years) with an office visit for uncomplicated acute bronchitis during the baseline (2006-2007, 2007-2008, and 2008-2009) and intervention (2009-2010) periods. We focused the intervention implementation during the periods of highest visit volumes for bronchitis; therefore, the analysis was restricted to visits occurring between October 1 and March 31 in each year. All clinicians caring for patients diagnosed as having acute bronchitis participated in the analysis, including board-certified internal medicine and family practice physicians, nurse practitioners, physician assistants, and registered nurses. The clinicians at the control sites were not informed of the study's objectives. Waivers of informed consent from patients and clinicians were obtained from the health system's institutional review board.
Incident adult office visits with the diagnosis of acute bronchitis (ICD-9-CM code 466.0) or bronchitis not otherwise specified (ICD-9-CM code 490) during the specified periods were identified from the EHR. Although the Healthcare Effectiveness Data and Information Set measure for acute bronchitis is based solely on ICD-9-CM code 466.0, we included both codes 466.0 and 490 because it was previously found that a significant coding shift toward the use of code 490 had occurred within the study sites in efforts to improve performance on this measure.19 Incident acute bronchitis visits were defined as not having an office visit for any ARI during the previous 30 days. We defined a return visit as a subsequent visit occurring within 30 days of an incident visit. To calculate antibiotic prescription rates for uncomplicated acute bronchitis (the primary outcome for this study), we excluded visits by patients younger than 13 years and older than 64 years, as well as visits by patients having secondary diagnoses of chronic heart and lung disease, human immunodeficiency virus, or malignant neoplasm or patients having secondary ARI diagnoses for which antibiotic therapy might be indicated (sinusitis, pharyngitis, otitis media, and pneumonia) (Figure 2).
Information on electronic alert and SmartSet use is captured only at the medical record level, meaning that a count is registered each time an electronic alert fires or a SmartSet is opened. If a nurse or clinician closes the medical record to attend to another issue and then reopens the medical record, then the electronic alert or SmartSet registers an additional count. Therefore, electronic alert and SmartSet counts will overestimate the use at the patient visit or clinician level. All office visit and prescription data were extracted from the EHR by individuals who were unaware of the allocation of randomization across the study sites. Other data collected for each visit from the medical record included patient age and sex, vital sign measures, smoking status, and chest radiography ordering.
Generalized estimating equations and alternating logistic regression (PROC GENMOD in SAS; SAS Institute, Inc) were used to control for clustering of antibiotic prescribing patterns by practice site and by clinician using patient visits as the unit of analysis. These models also adjusted for patient and clinician characteristics (at the visit level) that demonstrated a statistically significant change (P < .05) within study arms between baseline and intervention periods, including clinician type, clinician specialty, smoking status, and vital sign abnormalities. Statistical significance of a change in antibiotic prescribing between periods was defined as P < .05 for the interaction term between intervention status and baseline vs intervention period. Adjusted odds ratios (95% CIs) for antibiotic treatment during the intervention period were calculated from variable estimates. As a secondary analysis, we also examined changes in antibiotic prescription rates at the clinician level to assess the variation in clinician prescribing behavior change. We restricted this analysis to clinicians with at least 10 patient visits in each study period to achieve more precise estimates of baseline and intervention prescription rates; these included 31 of 68 PDS clinicians (45.6%), 26 of 41 CDS clinicians (63.4%), and 27 of 46 control clinicians (58.7%). All statistical analyses were performed using a software program (Statistical Application Program, release 9.2; SAS Institute, Inc).
This study was approved by the institutional review boards at the participating clinical sites, at the University of Pennsylvania, and at the University of California, San Francisco. The study was registered with ClinicalTrials.gov (Identifier: NCT00981994) before enrolling patients.
A total of 9808 incident visits for uncomplicated acute bronchitis took place during the baseline winter periods, and 6242 incident visits occurred during the intervention winter period (October 1, 2009, through March 31, 2010). The number of visits and proportion of ARIs diagnosed as uncomplicated acute bronchitis remained stable across study sites between the baseline and intervention periods. The proportions of total ARIs diagnosed as uncomplicated acute bronchitis during the baseline period and the intervention period, respectively, were 9.0% and 8.3% for PDS intervention sites, 7.8% and 8.0% for CDS intervention sites, and 10.3% and 9.5% for control sites. The comparison of patient and clinician characteristics associated with the final set of eligible study visits at PDS intervention sites, CDS intervention sites, and control sites demonstrated modest differences among study arms during the baseline period (all except fever and tachypnea were statistically significant given the large sample size) (Table). Comparing baseline and intervention years within study arms, statistically significant changes (P < .05) were observed for several variables (clinician type, clinician specialty, smoking status, and proportion of visits with fever or tachypnea), which we subsequently included in the multivariable analysis of the intervention effects.
Compared with the baseline period, the percentage of adolescents and adults prescribed antibiotics for uncomplicated acute bronchitis during the intervention period decreased at the PDS intervention sites (from 80.0% to 68.3%) and CDS intervention sites (from 74.0% to 60.7%) but increased slightly at the control sites (from 72.5% to 74.3%). After controlling for patient temperature, respiratory rate, smoking status, clinician type, clinician specialty, and clustering of observations by clinician and by practice site, the differences for the intervention sites were statistically significant from the control sites (P = .003 for control sites vs PDS intervention sites and P = .01 for control sites vs CDS intervention sites) but not between themselves (P = .67 for PDS intervention sites vs CDS intervention sites) (Figure 3). The adjusted odds ratios for antibiotic treatment during the intervention period compared with the baseline period were 0.57 (95% CI, 0.40-0.82) for PDS intervention sites, 0.64 (95% CI, 0.45-0.91) for CDS intervention sites, and 1.10 (95% CI, 0.85-1.43) for control sites.
To produce reliable estimates of prescription rates, we also measured changes in antibiotic prescription rates of individual clinicians in each group with a sufficient volume of patient visits during the baseline and intervention periods (≥10 visits each period). This subset of clinicians accounted for 81.8% of the total visits. The mean change in antibiotic prescription rates of these clinicians in the baseline and intervention periods was similar to the change based on the patient-level analysis (Figure 4). However, a significant proportion (about one-third) of clinicians reduced antibiotic prescription rates by more than 20% at both types of intervention sites.
Return visit rates (an office visit ≤30 days from an incident visit for uncomplicated acute bronchitis) increased modestly at all study sites and were not significantly different among study sites (Table). The proportion of patients diagnosed as having uncomplicated acute bronchitis at the incident visit who were subsequently diagnosed as having pneumonia at a return visit was low (range, 0.5%-1.5%). Similarly, subsequent emergency department visits and hospital admissions were rare across all sites and periods.
We recorded 11 827 occasions when an electronic alert fired during the check-in process (ie, for a chief symptom of cough). On 4789 occasions, the patients were given an educational brochure about appropriate antibiotic use. To place these electronic alert firings into context, during this period the CDS intervention sites provided care to 12 082 adolescent and adult patients who were diagnosed as having an ARI, and 2582 patients were diagnosed as having any type of bronchitis. For SmartSet use, the EHR system recorded 819 occasions when the SmartSet was opened, representing 26 of 43 clinicians on staff at the CDS intervention sites during the intervention period.
In this cluster randomized trial comparing the effectiveness of different implementation strategies for delivering clinical algorithm–based decision support for acute cough illness, we found that printed and computer-assisted approaches were equally effective at improving antibiotic treatment of uncomplicated acute bronchitis. No significant differences were observed in alteration of return visits between the baseline and intervention periods or among the study arms, suggesting that the application of the clinical algorithm and the resulting decrease in antibiotic treatment were not associated with adverse clinical consequences. In aggregate, these findings support the wider dissemination and use of this clinical algorithm to help reduce the overuse of antibiotics for acute bronchitis in primary care.
Our results demonstrate that conventional (noncomputerized) methods of implementing decision support for specific treatment decisions may be as effective as approaches that use computerized decision support, although this single finding cannot be generalized to all decision support interventions. Review of the electronic utilization data shows that the CDS approach was not heavily used by the physicians at those sites and may have contributed to the fact that it did not lead to greater levels of improvement compared with the traditional print-based decision support. A study20 using a CDS tool similar to that used in our study (but targeting all ARIs instead of just cough illness) showed little use and no overall effect on antibiotic prescribing behavior, whereas an ARI decision support tool delivered through personal digital assistant devices resulted in improvement in another study.21 The key finding from our study is that, when coupled with other traditional patient and physician education materials, both PDS and CDS strategies can achieve improvement. The choice of specific strategy should be guided by local site resources and competing priorities, not by an inherent belief that one strategy is superior to another. While we do not believe that it is feasible for every algorithm to be subjected to this type of comparative evaluation of implementation strategies, future comparative effectiveness research should help create a framework for identifying the most effective platforms or modes for delivering this type of decision support.
Several limitations should be considered in the interpretation of our findings. This trial was conducted in an integrated health care delivery system with a comprehensive EHR in place for several years before our study. The study sites were small- to medium-sized rural and semirural practices located outside of major metropolitan areas. One possible concern is that attention to the problem of antibiotic overprescribing for acute bronchitis could have led physicians to shift their use of diagnostic codes to more antibiotic-appropriate codes. However, such shifts have not been detected during prior interventional trials.8-10 It is possible that the effect of the intervention across a larger proportion of clinicians may take more time than one winter period. Longer follow-up periods may show a greater decline in antibiotic prescription rates or a separation in the relative effectiveness of the PDS and CDS strategies. Conversely, longer follow-up periods could also show a regression of the intervention effects. Because the decision support tools were embedded within a multidimensional intervention strategy and were evaluated prospectively, we cannot determine how much of the improvement in prescribing at PDS and CDS intervention sites was due to patient education, physician education, or clinical champions or simply resulted from the knowledge that they were being monitored. However, combined patient and clinician educational interventions have been the standard approach to improving antibiotic use for acute bronchitis, and no studies to date have successfully improved prescribing patterns with single-component interventions.
In conclusion, an evidence-based algorithm to guide management of acute bronchitis can reduce the overuse of antibiotics in primary care settings, but the mode of implementation does not seem to influence the magnitude of effect. Studies of computer-assisted decision support tools that do not include a comparison with more traditional implementation strategies may significantly overestimate the value of this type of decision support.
Correspondence: Ralph Gonzales, MD, MSPH, Division of General Internal Medicine, Department of Medicine, University of California, San Francisco, 3333 California St, PO Box 1211, San Francisco, CA 94118 (email@example.com).
Accepted for Publication: September 12, 2012.
Published Online: January 14, 2013. doi:10.1001/jamainternmed.2013.1589
Author Contributions: Dr Gonzales had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Gonzales, Anderer, McCulloch, Bloom, and Metlay. Acquisition of data: Anderer, Stahl, Yefko, and Molecavage. Analysis and interpretation of data: Gonzales, Anderer, McCulloch, Maselli, Bloom, Graf, and Metlay. Drafting of the manuscript: Gonzales, McCulloch, Maselli, and Metlay. Critical revision of the manuscript for important intellectual content: Gonzales, Anderer, McCulloch, Maselli, Bloom, Graf, Stahl, Yefko, and Metlay. Statistical analysis: Gonzales, McCulloch, and Maselli. Obtained funding: Gonzales and Metlay. Administrative, technical, and material support: Anderer, Bloom, Graf, Stahl, Yefko, Molecavage, and Metlay. Study supervision: Anderer, Bloom, and Graf.
Conflict of Interest Disclosures: Dr Gonzales serves as a medical adviser for Phreesia, Inc, a company that provides computerized patient check-in services. Dr Bloom serves on the Merck Speakers Bureau for the topic of patient-centered medical homes.
Funding/Support: This study was supported by grant R01CI000611 from the Centers for Disease Control and Prevention (Drs Gonzales and Metlay).
Role of the Sponsors: The sponsor had no role in the actual design or conduct of the study; in the collection, management, analysis, or interpretation of the data; or in the preparation, review, or approval of the manuscript.
Create a personal account or sign in to: