Clinical Decision Support and Appropriateness of Antimicrobial Prescribing: A Randomized Trial | Clinical Decision Support | JAMA | JAMA Network
[Skip to Navigation]
Sign In
Figure 1. Relative Change in Prescribing Rates in Clinical Decision Support System (CDSS), Community Intervention Alone (CI-Alone), and Nonstudy Communities
Image description not available.

A, Relative change in rate of total antimicrobial prescriptions per community population size during each postintervention year compared with the preintervention year derived using a multilevel Poisson regression model fitted to retail pharmacy data. B, Relative change in rate of macrolide prescriptions per community population size during each postintervention year compared with the preintervention year derived using a multilevel Poisson regression model fitted to retail pharmacy data. Error bars indicate 95% confidence intervals.

Figure 2. Relative Change in Prescribing Antimicrobial Agent for Visits in the “Never Indicated” Category
Image description not available.

Relative change in probability of prescribing an antimicrobial agent for visits in the “never indicated” category during the postintervention period compared with the preintervention period, based on fitting a multilevel logistic regression model to the chart review data. Horizontal bars represent study group estimates and circles represent each community, calculated as the group estimate plus community-specific residual.

Table 1. Characteristics of Communities Participating in the Randomized Trial and 6 Nonstudy Communities Selected for Comparison
Image description not available.
Table 2. Measurement of Diffusion of Intervention
Image description not available.
Table 3. Observed Antimicrobial Prescribing Rates by Study Arm and Year*
Image description not available.
Table 4. Multilevel Multinomial Logistic Regression Model for Selection of Drug Class at Visits for Which an Antimicrobial Agent Was Prescribed
Image description not available.
Table 5. Change in Odds of Prescribing an Antimicrobial Drug According to the Number of Case-Specific Algorithms Completed by the Clinician*
Image description not available.
Original Contribution
November 9, 2005

Clinical Decision Support and Appropriateness of Antimicrobial Prescribing: A Randomized Trial

Author Affiliations

Author Affiliations: VA Salt Lake City Health Care System (Drs Samore and Rupper); Department of Internal Medicine, University of Utah, Salt Lake City (Drs Samore, Bateman, Alder, Rubin, Stults, Rupper, and Stevenson, Mssrs Stoddard, Haddadin, and Ms Williamson); HealthInsight, Salt Lake City (Dr Bateman and Ms Donnelly); and Qualis Health, Boise, Idaho (Drs Hannah and Stevenson). Dr Stevenson is now with the Ohio State University College of Medicine, Columbus.

JAMA. 2005;294(18):2305-2314. doi:10.1001/jama.294.18.2305

Context The impact of clinical decision support systems (CDSS) on antimicrobial prescribing in ambulatory settings has not previously been evaluated.

Objective To measure the added value of CDSS when coupled with a community intervention to reduce inappropriate prescribing of antimicrobial drugs for acute respiratory tract infections.

Design, Participants and Setting Cluster randomized trial that included 407 460 inhabitants and 334 primary care clinicians in 12 rural communities in Utah and Idaho (6 with 1 shared characteristic and 6 with another), and a third group of 6 communities that served as nonstudy controls. The preintervention period was January to December 2001 and the postintervention period was January 2002 to September 2003. Acute respiratory tract infection diagnoses were classified into groups based on indication for antimicrobial use. Multilevel regression methods were applied to account for the clustered design.

Intervention Six communities received a community intervention alone and 6 communities received community intervention plus CDSS that were targeted toward primary care clinicians. The CDSS comprised decision support tools on paper and a handheld computer to guide diagnosis and management of acute respiratory tract infection.

Main Outcome Measure Community-wide antimicrobial usage was assessed using retail pharmacy data. Diagnosis-specific antimicrobial use was compared by chart review.

Results Within CDSS communities, 71% of primary care clinicians participated in the use of CDSS. The prescribing rate decreased from 84.1 to 75.3 per 100 person-years in the CDSS arm vs 84.3 to 85.2 in community intervention alone, and remained stable in the other communities (P = .03). A total of 13 081 acute respiratory tract infection visits were abstracted. The relative decrease in antimicrobial prescribing for visits in the antibiotics “never-indicated” category during the post-intervention period was 32% in CDSS communities and 5% in community intervention-alone communities (P = .03). Use of macrolides decreased significantly in CDSS communities but not in community intervention–alone communities.

Conclusion CDSS implemented in rural primary care settings reduced overall antimicrobial use and improved appropriateness of antimicrobial selection for acute respiratory tract infections.

Trial Registration Identifier: NCT00235703.

Antimicrobial resistance is a serious public threat that is exacerbated by the gradual withdrawal of the pharmaceutical industry from new antimicrobial agent development.1 Overuse of antimicrobial agents fosters the spread of antimicrobial-resistant organisms.2,3 Despite recent trends that demonstrate reduced outpatient use of antimicrobial agents, prescribing continues to significantly exceed prudent levels.4-8 Approximately 50% of courses of ambulatory antimicrobial drugs are prescribed for patients with viral respiratory infections and therefore are not clinically indicated.9-12 Behavioral facilitators of antimicrobial overuse and barriers to prudent use operate on both clinicians and patients.13-16 Patient demand, perceived or actual, creates challenges that distinguish this problem from other quality-of-care concerns.17-20 A number of studies have tackled the dual drivers of antimicrobial overuse by coupling academic outreach to clinicians with educational programs directed toward patients.21-27

Herein, we describe a 12-community randomized trial in which 2 different strategies to enhance appropriate use of antimicrobials for acute respiratory tract infections were compared. One arm of the randomized trial received a community intervention alone, while the other received the community intervention plus a direct intervention with primary care clinicians. The clinician intervention incorporated stand-alone decision support tools on paper or a handheld personal digital assistant (PDA) for the management of acute respiratory tract infection at the point-of-care. Thus, the major purpose of the study was to measure the added value of a clinical decision support system (CDSS) when coupled with a community intervention.

Study Communities

The community was chosen as the unit of randomization because it was a definable entity and the logical foundation for implementing a population-based intervention. Eligible rural communities were those in Utah and Idaho that contained at least 1 primary care clinic and inpatient facility. Each was located in a nonmetropolitan area based on the definition from the US Office of Management and Budget—an area with a total population of less than 100 000, consisting of cities with populations no greater than 50 000 each. Three rural communities that were involved in other antimicrobial use interventions were excluded. The remaining eligible communities were divided according to state and population-size category (<25 000 and ≥25 000). From each state, 4 smaller (<25 000) communities and 2 larger (≥25 000) communities were randomly selected. Then, one larger community from each state was randomly assigned to the combined community intervention plus CDSS intervention (hereafter referred to as the CDSS arm) and the other to the community intervention–alone arm. Next, 2 smaller communities from each state were randomly assigned to the CDSS arm and the other 2 to the community intervention–alone arm. Altogether, each intervention arm comprised 6 communities; the 6 communities from each state were evenly divided between the CDSS and community intervention–alone arms. Six other communities eligible for participation, but not selected, served as a randomly chosen nonstudy reference group for the analysis of retail pharmacy data.

The characteristics of the 12 study communities and 6 nonstudy communities are shown in Table 1; community-level data on race and ethnicity were obtained from the 2000 Census and are included for descriptive purposes. The distribution of clinician types across the 3 groups of communities was similar. The communities were dispersed widely through the 2-state region, minimizing potential for contamination. Other types of interventions, such as change in pharmacy detailing, were not implemented in the study communities during the period of the study.

Approval for the study was granted from both the University of Utah and the Western Institutional Review Boards. Both approved a waiver of informed consent since deidentified existing data were used. Furthermore, both institutional review boards determined that the use of the decision support tools did not require informed consent, since clinicians were not required to follow the guidelines advocated by the tools.

Description of the Intervention

The intervention was disseminated in 2 waves with the first beginning in January 2002 and the second beginning in November 2002.

Community Intervention

The community intervention was explicitly designed to map to constructs within behavior change models.28-30 The initial launch of the community intervention on the topic of antimicrobial resistance consisted of introductory meetings with community leaders, news releases in the print media, distribution of educational materials at pharmacies and physician offices, and a mailing to parents of children aged younger than 6 years. The educational materials included examination room posters and brochures about appropriate antimicrobial use and were printed in Spanish and English. The mailing consisted of a 5 × 7 postcard, with a refrigerator magnet attached to the upper right-hand corner. The key message was, “Do not treat viral infections with antibiotics.”

The second wave of the community intervention focused on 2 specific types of patient behaviors: (a) how to self-manage common respiratory tract infections; and (b) how to improve communication with your clinician. Individuals seeking care for acute respiratory tract infection were advised to ask their clinician the following 4 questions: (1) Is my illness caused by a virus or bacteria? (2) When should I expect to feel better? (3) When should I call your office or come back if I’m not feeling better? (4) What can I do at home to feel better? A spiral bound flip chart affixed to a magnet was developed as a guide to self-management of respiratory tract infections. This self-care guide was distributed by clinics at health fairs and special events, and through one-on-one interactions with community residents. In addition, a series of articles containing focused behavioral change messages were distributed to community newspapers.

CDSS Intervention

Three decision support tools were developed to cover a wide variety of acute respiratory tract infections including pharyngitis, otitis media, bronchitis, upper respiratory tract infection, sinusitis, pneumonia, croup, and influenza. Two versions were paper-based and 1 was programmed on a PDA.31 Three different formats of the tools were made available in order to give clinicians a choice and thereby, enhance their willingness to participate. One paper version was a patient-initiated chart-documentation tool on which the patient circled answers to questions about specific symptoms. The layout facilitated rapid interpretation of responses by the clinician and guided treatment appropriate for the patient’s diagnosis. The other paper version was an easy-to-use graphical flowchart also designed to lead the clinician to the correct diagnosis and treatment options according to precise criteria.

The PDA-based CDSS generated diagnostic and therapeutic recommendations on the basis of patient-specific information that was input about the suspected diagnosis, such as the presence or absence of specific symptoms and signs. Therapeutic recommendations included over-the-counter medications for symptom control as well as prescription antimicrobials. For pediatric patients, the advice was customized to the patient’s weight and age. For cases of pneumonia, the system also calculated the patient’s pneumonia severity index score.32

The decision support tools were introduced to primary care clinicians through educational lectures, small group meetings, and one-on-one interactions between primary care clinicians and physician members of the study team. Primary care clinicians were defined for this study as emergency department clinicians, family practice physicians, internists, pediatricians, nurse practitioners, and physician’s assistants. The following steps were undertaken in each community: (1) a local clinician contact was identified and solicited to lend support to the project; (2) an outline of the initiative was communicated at the local hospital’s medical staff meeting; (3) all of the primary care clinicians in the community were invited to a continuing medical education session (that covered the threat of antimicrobial resistance,33 the problem of overuse of antimicrobials, and evidence-based guidelines for the management of acute respiratory tract infections31,34-40) that was delivered jointly by an infectious diseases specialist and a primary care physician on the study team; and (4) primary care clinicians in the community were then recruited to participate in the use of case-specific algorithms, with their choice of format.

Clinicians were asked to use the algorithms on at least 200 consecutive patients with acute respiratory tract infections. Completed paper algorithms were retrieved from the clinicians; PDA cases were transmitted electronically. To offset any additional time needed during patient visits, a remuneration of $3 per case was given to participants up to 200 cases per clinician. Alternatively, clinicians were allowed to keep the PDA if that modality was selected.

Follow-up meetings were held with clinicians between November 2002 and December 2002 to give feedback on community-level antimicrobial prescribing data from the first year, and to recruit clinicians who did not previously participate. This session included a presentation that role modeled communication and management of patient expectations during acute respiratory tract infection visits. Data on clinician-specific prescribing patterns were not yet available at the time of these visits.

Measurement of Antimicrobial Use

Retail pharmacy volume and chart review were the 2 sources of information about antimicrobial prescribing that were used. Projected retail pharmacy antimicrobial prescriptions mapped to the 12 study communities and 6 nonstudy communities according to ZIP codes of clinicians were purchased from a company that specializes in providing data resources to pharmaceutical companies (IMS Health Inc; Xponent database).22 These data, which were derived from transaction records, were supplied as the number of new prescriptions per month per community, divided by antimicrobial drug name, formulation, and strength. The mean coverage of pharmacies in the study communities during the duration of the study period was 79%.

Chart review was performed only in the 12 study communities. Medical records in primary care clinician offices were randomly pulled for review, regardless of the clinician’s participation in the use of CDSS. Case-specific use of CDSS tools was not identifiable from chart review.

Using a detailed data dictionary and guidance manual, nurses trained in chart review extracted information about each acute respiratory tract infection visit. Symptoms, signs, diagnoses, test results, and medications associated with these visits were coded by applying precise criteria. Data were directly entered into a computer-based chart abstraction tool, which included error-trapping logic and data entry rules. Inter-rater reliability testing was performed to ensure consistency of coding.

The diagnosis recorded by the primary care clinician and the condition assigned by the reviewer on the basis of documented symptoms, signs, and test results were cataloged. Discrepancies between clinician and reviewer assessment were resolved by selecting the diagnosis most supportive of an indication for antimicrobial therapy. The diagnoses were categorized into 3 groups according to appropriateness of antibiotic use with labels stating: (1) “never indicated” (acute bronchitis and colds/upper respiratory tract infection); (2) “sometimes indicated” (sinusitis and uncharacterized otitis media or pharyngitis); and (3) “always indicated” (streptococcal pharyngitis, acute otitis media, and pneumonia).31,41,42 Only 1% of reviewed visits had clinician and reviewer diagnoses that mapped to discordant antibiotic indication groups.

The number of patients with acute respiratory tract infection reviewed per clinician varied according to the number of primary care clinicians practicing in the community (<15, 40 charts each; 15-30, 30 charts each; >30, 20 charts each). This sample size was designed to achieve an 80% statistical power to detect an absolute difference of 6% in prescribing of antimicrobials between the 2 arms of the study, allowing for an intra-cluster correlation of .05.43

Statistical Analysis

The prespecified analytic plan was to use mixed (multilevel) regression models to compare intervention arms, accounting for the clustered study design.44,45 Second order penalized quasi-likelihood was used for parameter estimation.46 Confidence intervals and P values were calculated from the Wald statistic. P values less than .05 were considered significant. The statistical software programs of MLwiN 2.0 (Centre for Multilevel Modeling, London, England) and Stata 8.0 (Stata Corporation, College Station, Tex) were used.

Multilevel modeling was selected as the primary method of analysis because the target inference was to estimate the expected community-specific change in antimicrobial use.47,48 To assess robustness of the results with respect to choice of statistical model, secondary analyses were conducted using generalized estimating equations to generate population-averaged estimates of effect.49 As predicted by theory,50,51 generalized estimating equations regression methods produced intervention β coefficients that were modestly attenuated compared with the multilevel regression models, although with narrower confidence intervals. None of the principal conclusions about statistical significance were altered. Only the results of the multilevel models are presented here.

Retail pharmacy data were examined by multilevel Poisson regression using the number of antimicrobial prescriptions per month as the first level and the community as the second level. Extra Poisson variation was permitted. The logarithm of the community population size for each year was incorporated as an offset. The community population size included the central community and its adjacent satellite communities within the same ZIP code corresponding to pharmacy locations. The 2001 to 2003 populations were extrapolated from the 2000 US Census Bureau ZIP code–level counts by applying a growth multiplier, derived by linearly interpolating 1990 and 2000 community level census data. A sensitivity analysis to determine whether varying these population assumptions influenced the interpretation of the intervention effect was performed.

The main terms in the model were study group and time period. The 3 groups of communities were CDSS, community intervention alone, and nonstudy. In this analysis, the 3 time intervals were before the intervention (January 2001-September 2001), first year after the intervention (January 2002-September 2002), and the second year after the intervention (January 2003-September 2003) to maintain an equal number of months for each period. The interaction terms study group × time period after the intervention were used to test the null hypothesis that the change in prescribing in CDSS communities equaled the change in prescribing in community intervention–alone communities. The interaction and intercept terms comprised both fixed and random components to accommodate variation across communities. A sine function was included to account for seasonal fluctuation in the rate of antimicrobial prescribing.52 This variable was a strong predictor of rate of antimicrobial prescribing but its inclusion or removal did not influence the estimate of the intervention effect. Separate models were fit to assess effects on total antimicrobial prescriptions and antimicrobial class-specific prescriptions.

Chart review data were analyzed using multilevel logistic regression to model the probability of being prescribed an antibiotic. Level 1 was the respiratory tract infection visit, level 2 was the clinician, and level 3 was the community. When an individual patient experienced multiple acute respiratory tract infection visits within a 1-month period, only the first visit was analyzed. The 2 study groups and 2 time periods, before the intervention (January 2001-December 2001) and after the intervention (January 2002-September, 2003), were compared as main effect terms; the 2 periods after the intervention were combined into 1 interval to facilitate model convergence on stable estimates. Intercept and post-intervention terms included both fixed and random components. The interaction term CDSS group × postintervention, was used to statistically test the null hypothesis that the change in probability of prescribing an antibiotic for infections belonging to the diagnostic category antibiotics never indicated, was equal in the 2 groups of communities. Inserting the fixed components of the model into the logistic equation for predicted probabilities, the proportions of patients who received antimicrobial drugs within each study arm and intervention period were estimated. This allowed communication of the results in the more readily understood terms of relative risk and risk difference rather than odds.53

Selection of drug class, given that an antimicrobial was prescribed at the acute respiratory tract infection visit, was also examined. This model comprised acute respiratory tract infection diagnoses belonging to all 3 categories. Antimicrobial agents were grouped into 4 classes: penicillins, macrolides, cephalosporins, and other. Visits for which an antimicrobial was not prescribed were excluded from this analysis. Penicillin-type drugs, as the most commonly prescribed class, represented the referent category. Model terms were similar to those described previously. Market share percentages (the percent of total antimicrobials accounted for by each drug class) were derived as predicted values from this multinomial-type model.

Multilevel logistic regression was also used to assess whether the magnitude of use of the algorithms was associated with decreased probability of prescribing an antibiotic for diagnoses belonging to the never indicated category within the CDSS communities. The change in odds of prescribing an antibiotic in the time period after the intervention according to the number of algorithms completed was estimated from a model containing: (a) the main effect terms of quartile of number of algorithms used and the period after the intervention; and (b) interaction terms of quartile × the period after the intervention. Models were also constructed that included the interaction term of algorithm use as a continuous variable × the period after the intervention as a test of linear trend.

Exposure to the Intervention

Community Intervention. A total of 83% to 100% of clinics and pharmacies in each community participated in the distribution of educational materials including posters and brochures (Table 2). Between January 2003 and September 2003, approximately 30 000 self-care guides were distributed through clinics, health fairs, and other special community events.

CDSS Intervention. Forty-five percent (80 of 176) of the primary care clinicians in the CDSS communities attended at least 1 of the annual continuing medical education sessions and 71% used the decision-support tools (Table 2). Fifty three percent of CDSS users began during the first intervention year and the rest initiated participation during the second year. All together, 54% used the PDA version only, 23% used paper algorithms only, and 24% used a mix of PDA and paper algorithms. A total of 20 727 respiratory tract infection cases were completed; 13 424 were PDA, 6354 were paper graphical flowcharts, and 949 were patient-initiated chart documentation forms.

Changes in Antimicrobial Use

Retail Pharmacy Data. The estimated community-specific rates of antimicrobial prescribing during the year before the intervention ranged from 26 to 198 prescriptions per 100 person-years. Prescribing rates of antimicrobials before the intervention were not significantly different across study arms (Table 3).

Rates of antimicrobial prescribing did not change significantly during the first intervention year. In CDSS and community intervention–alone communities, a nonsignificant decrease of 1% and an increase of 3% from baseline were observed; in nonstudy communities, prescribing rates decreased by 3% compared with baseline. During the second intervention year, prescribing rates in CDSS communities decreased 10% from baseline, whereas in the community intervention–alone communities and nonstudy communities, prescribing rates in 2003 increased by 1% and 6%, respectively (Figure 1). Within CDSS communities, the overall antimicrobial prescribing rate declined by an absolute amount of 0.09 prescriptions per person-year between baseline and the second-intervention year (Table 3). This translated to an expected reduction of 93 antimicrobial prescriptions per month in a rural community with a population size equal to the mean of the CDSS group. The test of whether prescribing decreased more in CDSS than community intervention–alone communities was statistically significant (P  = .03).

In CDSS communities, rates of prescribing of antimicrobials belonging to the macrolide class decreased by 12% (P = .002) during the first intervention year and by 28% (P<.001) during the second intervention year, but remained stable in the other communities (Figure 1). In total, 83% of the macrolide prescriptions were for azithromycin. Prescriptions of other drug classes—cephalosporins and penicillins—declined by 6% and 7%, respectively, in the CDSS communities in 2003 compared with 2001; neither change reached statistical significance.

Chart Review. Overall, charts of 79% of primary care clinicians in the 12 communities were reviewed. A total of 13 081 acute respiratory tract infection visits that took place between January 2001 and September 2003 were abstracted. Fifty-five percent of visits were pediatric and 45% adult. The 5 most common conditions were colds/upper respiratory tract infection (25%), otitis media (18%), pharyngitis (18%), sinusitis (16%), and bronchitis (12%). The distribution of these diagnoses did not change significantly during the 3-year period in either the CDSS or community intervention–alone communities. At baseline, neither the frequency of prescribing of antimicrobial drugs or the distribution of antimicrobial drug classes differed significantly between groups of communities.

Fitting the multilevel model to the data, which was the test of whether antimicrobial prescribing for visits in the antibiotics never indicated category decreased more in the CDSS arm than the community intervention–alone arm and was statistically significant (P  = .03). The estimated relative risk reduction54 was 32% in CDSS communities and 5% in community intervention–alone communities (Figure 2). This corresponded with a decrease in antimicrobial prescribing for visits in the never indicated category from 35% to 24% (11% difference) in CDSS communities and from 40% to 38% (2% difference) in community intervention alone–communities. Prescribing of an antimicrobial agent for diagnoses in the sometimes indicated or always indicated groups did not decline in either study group.

Based on an analysis of all acute respiratory tract infections, the proportion of antimicrobial drugs that were macrolides decreased in CDSS communities. In the multinomial model, the reduction in odds of prescribing macrolides relative to penicillins was 50% in CDSS communities (P = .001) (Table 4). This corresponded with a decreased market share for macrolides in CDSS communities from 28% to 17% and increased market share for penicillins from 53% to 63%. In contrast, the odds of macrolide use relative to penicillins increased by 12% in community intervention communities (P>.22), corresponding with a stable market share of penicillins (52%-51%) and a slight increase for macrolides (22%-25%). The fraction of antimicrobial agents represented by cephalosporins did not change significantly in either group of communities.

Correlation Between Use of CDSS and Change in Antimicrobial Prescribing

The magnitude of decrease in antimicrobial prescribing for diagnoses belonging to the category antibiotics never indicated correlated with the number of case-specific algorithms used. Within CDSS communities, clinicians in the highest 2 quartiles of algorithms completed showed a greater decrease in antimicrobial use than clinicians who did not use the tools (linear trend, P = .034) (Table 5).


This trial demonstrated the feasibility, uptake, and benefit of stand-alone, portable CDSS tools for acute respiratory tract infections in rural primary care settings. The CDSS decreased unnecessary use of antimicrobial agents for viral respiratory tract infections and improved antimicrobial agent selection. The macrolide class of drugs, which was not considered first line for any of the common acute upper respiratory tract infections,37,40 declined the most in use. This experience contrasts with US and European data that have demonstrated a marked upward trend in macrolide prescribing during recent years.8,11

The aim of recruiting clinicians to use the algorithms for a minimum number of cases was to interfere with automated, inappropriate antimicrobial decision making55 and to give practical training in the application of an evidence-based approach to respiratory tract infection management. In an intention-to-treat analysis, it is not possible to define the mechanism by which the intervention worked. However, the analysis of algorithm use supported the inference that its effect was related to the frequency of use. The results, when considered together, strongly suggest that the strategy of repetitive use of a diagnostic and treatment algorithm to ingrain new prescribing habits is a valuable component of a practice change intervention.

The decision-support tools generated case-specific recommendations on the basis of multiple items of individual patient data, a feature that differentiates this CDSS intervention from reference programs or guidelines that are not patient specific and therefore do not meet criteria for CDSS.56-60 It was necessary for all patient-specific data to be manually entered into the PDA because the system was not integrated with a practice management system or electronic medical record. This inconvenience was mitigated by the design of the program, in which treatment recommendations were rapidly displayed once minimum criteria for reaching an end node of the decision logic were met.57,58 Electronic prescribing programs create the possibility of integrating the respiratory tract infection decision-support logic more seamlessly into clinic workflow, even in the absence of fully computerized medical records.61-63

Remote rural communities are settings where CDSS may be particularly beneficial because of limited access to consultation and subspecialty services.64 Moreover, the geographic isolation of rural communities is an advantage for community randomized trials because it is easier to define distinct populations. However, the rural setting also poses particular challenges.65-67 The large distances between communities limited the opportunity for frequent, direct contacts between rural clinicians and physician members of our study team. Rural clinicians may be more likely than their urban counterparts to have social contacts with patients outside of the therapeutic relationship.68,69 Feedback from rural clinicians suggested that perception of patient demand and fear of public censure in the event of a medical mishap may operate at a more intense level in rural communities.

The lack of significant decrease in antimicrobial prescribing in community intervention–alone communities should not necessarily be construed as evidence that this component lacked any effect.70 The absence of a CDSS–alone arm precluded determination of whether the community activities enhanced the effect of the clinician intervention. Awareness on the part of clinicians of the messages being delivered to community residents may have facilitated acceptance of the CDSS and willingness to change antimicrobial prescribing practices. When an intervention is directed only at community residents, behavior change may take longer than the period of this study to unfold.

This study had certain limitations. The intervention did not address with equal intensity all of the obstacles to changing antimicrobial drug prescribing.71,72 The feedback of information about antimicrobial prescribing given between the first and second year of the intervention did not include clinician-specific data73 because it was not possible to perform chart review at the study mid point. Certain parts of the algorithms, such as the evaluation of severity of symptoms in sinusitis, required subjective interpretation on the part of the diagnostician. Increasing the specificity of rules may be useful to reduce practice variation but conversely may decrease likelihood of adoption.74 A shortcoming of the study design was the inclusion of a relatively small number of communities, a reflection of resource restraints. An asset of the study was that the communities were randomly selected from those eligible and not self-selected on the basis of willingness to participate.

Changes in antimicrobial prescribing as assessed by chart review and retail pharmacy data were consistent. The measure based on retail pharmacy prescriptions corresponded most closely with community-level antimicrobial selection pressure. Since acute respiratory tract infections are the dominant indication for outpatient antimicrobial use, this metric was expected to correlate both with the propensity of prescribing antimicrobials for respiratory tract infection and with the volume of visits for respiratory tract infections.75 Thus, it had the potential to detect variation in care-seeking behavior for respiratory tract infection symptoms, a possible effect of the community intervention. However, because it was not diagnosis- or individual patient–specific, it was less useful for examining the quality of antimicrobial prescribing and for assessing specific targets of practice change.

An unresolved question is whether the modest decrease in total antimicrobial prescriptions and more substantial reduction in macrolide use induced by the CDSS intervention was sufficient to lessen selection of resistant pneumococci and other bacteria in community populations.2,3,76 Decreased prevalence of resistant organisms may not necessarily accompany lowered antimicrobial consumption, in part because resistant organisms have an ability to develop compensatory mutations that ameliorate the fitness costs of resistance.77 More potent interventions that sustain greater improvements in antimicrobial use may be needed to adequately control antimicrobial resistance.

Back to top
Article Information

Corresponding Author: Matthew H. Samore, MD, Department of Internal Medicine, University of Utah School of Medicine, 50 N Medical Dr, AC230A SOM, Salt Lake City, UT 84132 (

Author Contributions: Dr Samore had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Samore, Bateman, Alder, Donnelly, Stoddard, Rubin, Stults, Stevenson.

Acquisition of data: Samore, Bateman, Alder, Hannah, Donnelly, Haddadin, Rubin.

Analysis and interpretation of data: Samore, Bateman, Alder, Donnelly, Stoddard, Rubin, Williamson, Rupper.

Drafting of the manuscript: Samore, Alder, Rupper.

Critical revision of the manuscript for important intellectual content: Samore, Bateman, Alder, Hannah, Donnelly, Stoddard, Haddadin, Rubin, Williamson, Stults, Stevenson.

Statistical analysis: Samore, Alder, Stoddard, Williamson.

Obtained funding: Samore, Alder, Donnelly, Rubin.

Administrative, technical, or material support: Hannah, Donnelly, Haddadin, Stults, Rupper, Stevenson.

Study supervision: Samore, Bateman, Hannah, Haddadin, Stevenson.

Financial Disclosures: None reported.

Funding/Support: Funding was provided by the Centers for Disease Control and Prevention, grant RS1 CCR820631. The development of the handheld personal digital assistant decision support tool described in this study was supported by Centers for Disease Control and Prevention grant RS1 CCR820631. It was designed by study investigators working with TheraDoc (Salt Lake City, Utah) as the technology partner. The program was created for research purposes only and is not currently or anytime in the future available for commercial sale.

Role of the Sponsor: The funding agency had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; or preparation, review, or approval of the manuscript.

Acknowledgment: We thank Summer Borla, BS, RN, for leading the chart abstraction and Terri Rose for study coordination and field work (HealthInsight, Salt Lake City). We thank Scott Engelstad, BSN, Kathryn Gibson, MS, Laurie Johnson, MPH/HSA, Amiee Maxwell, BS, Wendi Morales, BS, and Amyanne Wuthrich-Reggio, MS, for study coordination and field work (University of Utah, Salt Lake City). Compensation for these contributions was provided solely by the study grant from the Centers for Disease Control and Prevention.

Spellberg B, Powers JH, Brass EP, Miller LG, Edwards JE Jr. Trends in antimicrobial drug development: implications for the future.  Clin Infect Dis. 2004;38:1279-128615127341Google ScholarCrossref
Goossens H, Ferech M, Vander Stichele R, Elseviers M. Outpatient antibiotic use in Europe and association with resistance: a cross-national database study.  Lancet. 2005;365:579-58715708101Google Scholar
Lipsitch M, Samore MH. Antimicrobial use and antimicrobial resistance: a population perspective.  Emerg Infect Dis. 2002;8:347-35411971765Google ScholarCrossref
McCaig LF, Besser RE, Hughes JM. Trends in antimicrobial prescribing rates for children and adolescents.  JAMA. 2002;287:3096-310212069672Google ScholarCrossref
Finkelstein JA, Stille C, Nordin J.  et al.  Reduction in antibiotic use among US children, 1996-2000.  Pediatrics. 2003;112:620-62712949293Google ScholarCrossref
Steinman MA, Gonzales R, Linder JA, Landefeld CS. Changing use of antibiotics in community-based outpatient practice, 1991-1999.  Ann Intern Med. 2003;138:525-53312667022Google ScholarCrossref
Besser RE. Antimicrobial prescribing in the United States: good news, bad news.  Ann Intern Med. 2003;138:605-60612667034Google ScholarCrossref
Kuyvenhoven MM, van Balen FA, Verheij TJ. Outpatient antibiotic prescriptions from 1992 to 2001 in the Netherlands.  J Antimicrob Chemother. 2003;52:675-67812951333Google ScholarCrossref
Gonzales R, Steiner JF, Sande MA. Antibiotic prescribing for adults with colds, upper respiratory tract infections, and bronchitis by ambulatory care physicians.  JAMA. 1997;278:901-9049302241Google ScholarCrossref
Nyquist AC, Gonzales R, Steiner JF, Sande MA. Antibiotic prescribing for children with colds, upper respiratory tract infections, and bronchitis.  JAMA. 1998;279:875-8779516004Google ScholarCrossref
McCaig LF, Besser RE, Hughes JM. Antimicrobial drug prescription in ambulatory care settings, United States, 1992-2000.  Emerg Infect Dis. 2003;9:432-43712702222Google ScholarCrossref
Cantrell R, Young AF, Martin BC. Antibiotic prescribing in ambulatory care settings for adults with colds, upper respiratory tract infections, and bronchitis.  Clin Ther. 2002;24:170-18211833830Google ScholarCrossref
Davey P, Pagliari C, Hayes A. The patient's role in the spread and control of bacterial resistance to antibiotics.  Clin Microbiol Infect. 2002;8:(suppl 2)  43-6812427207Google ScholarCrossref
Finch RG, Metlay JP, Davey PG, Baker LJ. Educational interventions to improve antibiotic use in the community: report from the International Forum on Antibiotic Resistance (IFAR) colloquium, 2002.  Lancet Infect Dis. 2004;4:44-5314720568Google ScholarCrossref
Harbarth S, Albrich W, Brun-Buisson C. Outpatient antibiotic use and prevalence of antibiotic-resistant pneumococci in France and Germany: a sociocultural perspective.  Emerg Infect Dis. 2002;8:1460-147012498664Google ScholarCrossref
Mainous AG III, Zoorob RJ, Oler MJ, Haynes DM. Patient knowledge of upper respiratory infections: implications for antibiotic expectations and unnecessary utilization.  J Fam Pract. 1997;45:75-839228917Google Scholar
Mangione-Smith R, McGlynn EA, Elliott MN, McDonald L, Franz CE, Kravitz RL. Parent expectations for antibiotics, physician-parent communication, and satisfaction.  Arch Pediatr Adolesc Med. 2001;155:800-80611434847Google ScholarCrossref
Stivers T, Mangione-Smith R, Elliott MN, McDonald L, Heritage J. Why do physicians think parents expect antibiotics? what parents report vs what physicians believe.  J Fam Pract. 2003;52:140-14812585992Google Scholar
Barden LS, Dowell SF, Schwartz B, Lackey C. Current attitudes regarding use of antimicrobial agents: results from physician's and parents' focus group discussions.  Clin Pediatr (Phila). 1998;37:665-6719825210Google ScholarCrossref
Macfarlane J, Holmes W, Macfarlane R, Britten N. Influence of patients' expectations on antibiotic management of acute lower respiratory tract illness in general practice: questionnaire study.  BMJ. 1997;315:1211-12149393228Google ScholarCrossref
Perz JF, Craig AS, Coffey CS.  et al.  Changes in antibiotic prescribing for children after a community-wide campaign.  JAMA. 2002;287:3103-310912069673Google ScholarCrossref
Belongia EA, Sullivan BJ, Chyou PH, Madagame E, Reed KD, Schwartz B. A community intervention trial to promote judicious antibiotic use and reduce penicillin-resistant Streptococcus pneumoniae carriage in children.  Pediatrics. 2001;108:575-58311533321Google ScholarCrossref
Gonzales R, Steiner JF, Lum A, Barrett PH Jr. Decreasing antibiotic use in ambulatory practice: impact of a multidimensional intervention on the treatment of uncomplicated acute bronchitis in adults.  JAMA. 1999;281:1512-151910227321Google ScholarCrossref
Schaffner W, Ray WA, Federspiel CF.  et al.  Improving antibiotic prescribing in office practice: a controlled trial of three educational methods.  JAMA. 1983;250:1728-17326350633Google ScholarCrossref
Finkelstein JA, Davis RL, Dowell SF.  et al.  Reducing antibiotic use in children: a randomized trial in 12 practices.  Pediatrics. 2001;108:1-711433046Google ScholarCrossref
Belongia EA, Naimi TS, Gale CM, Besser RE. Antibiotic use and upper respiratory infections: a survey of knowledge, attitudes, and experience in Wisconsin and Minnesota.  Prev Med. 2002;34:346-35211902851Google ScholarCrossref
Welschen I, Kuyvenhoven MM, Hoes AW, Verheij TJ. Effectiveness of a multiple intervention to reduce antibiotic prescribing for respiratory tract symptoms in primary care: randomised controlled trial.  BMJ. 2004;329:43115297305Google ScholarCrossref
Alder SC, Trunnell EP, White GL.  et al.  Reducing parental demand for antibiotics by promoting communication skills.  Am J Health Edu. 2005;36:132-139Google ScholarCrossref
Elder JP, Ayala GX, Harris S. Theories and intervention approaches to health-behavior change in primary care.  Am J Prev Med. 1999;17:275-28410606196Google ScholarCrossref
Rosenstock IM, Strecher VJ, Becker MH. Social learning theory and the health belief model.  Health Educ Q. 1988;15:175-1833378902Google ScholarCrossref
Rubin MA, Bateman K, Alder S, Donnelly S, Stoddard GJ, Samore MH. A multifaceted intervention to improve antimicrobial prescribing for upper respiratory tract infections in a small rural community.  Clin Infect Dis. 2005;40:546-55315712077Google ScholarCrossref
Dean NC, Bateman KA. Local guidelines for community-acquired pneumonia: development, implementation, and outcome studies.  Infect Dis Clin North Am. 2004;18:975-99115555835Google ScholarCrossref
Samore MH, Magill MK, Alder SC.  et al.  High rates of multiple antibiotic resistance in Streptococcus pneumoniae from healthy children living in isolated rural communities: association with cephalosporin use and intrafamilial transmission.  Pediatrics. 2001;108:856-86511581436Google ScholarCrossref
Finch RG, Low DE. A critical assessment of published guidelines and other decision-support systems for the antibiotic treatment of community-acquired respiratory tract infections.  Clin Microbiol Infect. 2002;8:(suppl 2)  69-9112427208Google ScholarCrossref
Gonzales R, Bartlett JG, Besser RE.  et al.  Principles of appropriate antibiotic use for treatment of uncomplicated acute bronchitis: background.  Ann Intern Med. 2001;134:521-52911255532Google ScholarCrossref
Gonzales R, Bartlett JG, Besser RE, Hickner JM, Hoffman JR, Sande MA. Principles of appropriate antibiotic use for treatment of nonspecific upper respiratory tract infections in adults: background.  Ann Intern Med. 2001;134:490-49411255526Google ScholarCrossref
Gonzales R, Bartlett JG, Besser RE.  et al.  Principles of appropriate antibiotic use for treatment of acute respiratory tract infections in adults: background, specific aims, and methods.  Ann Intern Med. 2001;134:479-48611255524Google ScholarCrossref
Rosenstein N, Phillips WR, Gerber MA, Marcy SM, Schwartz B, Dowell SF. The common cold–principles of judicious use of antimicrobial agents.  Pediatrics. 1998;101:S181-S184Google Scholar
Schwartz B, Marcy SM, Phillips WR, Gerber MA, Dowell SF. Pharyngitis–principles of judicious of antimicrobial agents.  Pediatrics. 1999;101:S171-S174Google Scholar
Dowell SF, Butler JC, Giebink GS.  et al.  Acute otitis media: management and surveillance in an era of pneumococcal resistance–a report from the drug-resistant Streptococcus pneumoniae Therapeutic Working Group.  Pediatr Infect Dis J. 1999;18:1-9[published erratum appears in Pediatr Infect Dis J. 1999;18:341].9951971Google Scholar
Gonzales R, Malone DC, Maselli JH, Sande MA. Excessive antibiotic use for acute respiratory infections in the United States.  Clin Infect Dis. 2001;33:757-76211512079Google ScholarCrossref
Gonzales R, Sande M. What will it take to stop physicians from prescribing antibiotics in acute bronchitis?  Lancet. 1995;345:665-6667885119Google ScholarCrossref
Campbell MK, Mollison J, Grimshaw JM. Cluster trials in implementation research: estimation of intracluster correlation coefficients and sample size.  Stat Med. 2001;20:391-39911180309Google ScholarCrossref
Raudenbush SW, Bryk AS. Hierarchical Linear Models: Applications and Data Analysis Methods. Thousand Oaks, Calif: Sage Publications; 2002
Bingenheimer JB, Raudenbush SW. Statistical and substantive inferences in public health: issues in the application of multilevel models.  Annu Rev Public Health. 2004;25:53-7715015912Google ScholarCrossref
Rodriquez G, Goldman N. Improved estimation procedure for multilevel models with binary responses: a case-study.  J R Stat Soc [Ser A]. 2001;164:339-355Google ScholarCrossref
Donner A, Klar N. Pitfalls of and controversies in cluster randomization trials.  Am J Public Health. 2004;94:416-42214998805Google ScholarCrossref
Murray DM, Varnell SP, Blitstein JL. Design and analysis of group-randomized trials: a review of recent methodological developments.  Am J Public Health. 2004;94:423-43214998806Google ScholarCrossref
Bellamy SL, Gibberd R, Hancock L.  et al.  Analysis of dichotomous outcome data for community intervention studies.  Stat Methods Med Res. 2000;9:135-15910946431Google ScholarCrossref
Neuhaus JM, Segal MR. Design effects for binary regression models fitted to dependent data.  Stat Med. 1993;12:1259-12688210825Google ScholarCrossref
Neuhaus JM. Statistical methods for longitudinal and clustered designs with binary responses.  Stat Methods Med Res. 1992;1:249-2731341660Google ScholarCrossref
Koch HJ, Jost D, Szecsey A, Raschka C. Seasonal variation of antibiotic and psycho-pharmacological drug prescriptions in a neuro-psychiatric hospital assessed by means of sine-wave models.  Eur J Clin Pharmacol. 2001;56:943-94511317485Google ScholarCrossref
McNutt LA, Wu C, Xue X, Hafner JP. Estimating the relative risk in cohort studies and clinical trials of common outcomes.  Am J Epidemiol. 2003;157:940-94312746247Google ScholarCrossref
Barratt A, Wyer PC, Hatala R.  et al.  Tips for learners of evidence-based medicine, 1: relative risk reduction, absolute risk reduction and number needed to treat.  CMAJ. 2004;171:353-35815313996Google Scholar
Hastie R. Problems for judgment and decision making.  Annu Rev Psychol. 2001;52:653-68311148321Google ScholarCrossref
Cimino JJ, Bakken S. Personal digital educators.  N Engl J Med. 2005;352:860-86215745975Google ScholarCrossref
Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success.  BMJ. 2005;330:76515767266Google ScholarCrossref
Hunt DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review.  JAMA. 1998;280:1339-13469794315Google ScholarCrossref
Sim I, Gorman P, Greenes RA.  et al.  Clinical decision support systems for the practice of evidence-based medicine.  J Am Med Inform Assoc. 2001;8:527-53411687560Google ScholarCrossref
Wyatt J, Spiegelhalter D. Field trials of medical decision-aids: potential problems and solutions.  Proc Annu Symp Comput Appl Med Care. 1991;:3-71807610Google Scholar
Bates DW, Ebell M, Gotlieb E, Zapp J, Mullins HC. A proposal for electronic medical records in U.S. primary care.  J Am Med Inform Assoc. 2003;10:1-1012509352Google ScholarCrossref
Bates DW, Gawande AA. Improving safety with information technology.  N Engl J Med. 2003;348:2526-253412815139Google ScholarCrossref
Bell DS, Cretin S, Marken RS, Landman AB. A conceptual framework for evaluating outpatient electronic prescribing systems based on their functional capabilities.  J Am Med Inform Assoc. 2004;11:60-7014527975Google ScholarCrossref
Dorsch JL. Information needs of rural health professionals: a review of the literature.  Bull Med Libr Assoc. 2000;88:346-35411055302Google Scholar
Parsons JE, Merlin TL, Taylor JE, Wilkinson D, Hiller JE. Evidence-based practice in rural and remote clinical practice: where is the evidence?  Aust J Rural Health. 2003;11:242-24814641222Google Scholar
Taylor CT, Byrd DC, Krueger K. Improving primary care in rural Alabama with a pharmacy initiative.  Am J Health Syst Pharm. 2003;60:1123-112912816022Google Scholar
Worrall G, Young B, Knight V. Inappropriate use of antibiotics for acute respiratory tract infections in a rural emergency department.  Can J Rural Med. 2005;10:86-8815842790Google Scholar
Scopelliti J, Judd F, Grigg M.  et al.  Dual relationships in mental health practice: issues for clinicians in rural settings.  Aust N Z J Psychiatry. 2004;38:953-95915555031Google ScholarCrossref
Jackson WC. When patients are normal people: strategies for managing dual relationships.  Prim Care Companion J Clin Psychiatry. 2002;4:100-10315014737Google ScholarCrossref
Gonzales R, Corbett KK, Leeman-Castillo BA.  et al.  The "minimizing antibiotic resistance in Colorado" project: impact of patient education in improving antibiotic use in private office practices.  Health Serv Res. 2005;40:101-11615663704Google ScholarCrossref
Mangione-Smith R, Stivers T, Elliott M, McDonald L, Heritage J. Online commentary during the physical examination: a communication tool for avoiding inappropriate antibiotic prescribing?  Soc Sci Med. 2003;56:313-32012473316Google ScholarCrossref
Cabana MD, Rand CS, Powe NR.  et al.  Why don't physicians follow clinical practice guidelines? a framework for improvement.  JAMA. 1999;282:1458-146510535437Google ScholarCrossref
Kiefe CI, Allison JJ, Williams OD, Person SD, Weaver MT, Weissman NW. Improving quality improvement using achievable benchmarks for physician feedback: a randomized controlled trial.  JAMA. 2001;285:2871-287911401608Google ScholarCrossref
Morris AH. Developing and implementing computerized protocols for standardization of clinical decisions.  Ann Intern Med. 2000;132:373-38310691588Google ScholarCrossref
Bauraind I, Lopez-Lozano JM, Beyaert A.  et al.  Association between antibiotic sales and public campaigns for their appropriate use.  JAMA. 2004;292:2468-247015562124Google Scholar
Lipsitch M. The rise and fall of antimicrobial resistance.  Trends Microbiol. 2001;9:438-44411553456Google ScholarCrossref
Austin DJ, Kristinsson KG, Anderson RM. The relationship between the volume of antimicrobial consumption in human communities and the frequency of resistance.  Proc Natl Acad Sci U S A. 1999;96:1152-11569927709Google ScholarCrossref