Use of Simple Heuristics to Target Macrolide Prescription in Children With Community-Acquired Pneumonia

Background  Macrolides are the first-line antibiotic treatment of community-acquired pneumonia (CAP). Owing to alarming resistance rates among invasive Streptococcus pneumoniae isolates, particularly in young children, macrolide use should be restricted to patients infected with susceptible pathogens, eg, Mycoplasma pneumoniae.

Objective  To develop a simple clinical prediction rule for identifying M pneumoniae as the cause of CAP in children.

Design and Setting  Prospective cohort study in 253 children with radiologically confirmed CAP in a walk-in clinic of a tertiary care hospital.

Main Outcome Measures  Mycoplasma infection, proven by results of antibody testing of paired serum samples (gold standard). We compared the area under the receiver operating characteristic curve (c statistic) of the following 2 prediction models: a scoring system derived from logistic regression analysis and a fast-and-frugal decision tree.

Results  Mycoplasma pneumoniae infection was confirmed in 32 (13%) of 253 children. A scoring system based on duration of fever and patient age yielded a c statistic of 0.84 (95% confidence interval [CI], 0.77-0.91), compared with that of the decision tree (c = 0.76 [95% CI, 0.70-0.83]). The scoring system identified 75% of all cases as being at high or very high risk for M pneumoniae infection; the decision tree, 72% at high risk. The scoring system would curtail macrolide prescriptions by 75%; the decision tree, by 68%.

Conclusions  In children with CAP, simple clinical decision rules identify patients at risk for M pneumoniae infection. At present US macrolide resistance rates among invasive S pneumoniae isolates, both rules increase the chance of prescribing effective first-line antibiotics compared with general macrolide administration.

THE MOST COMMON bacterial cause of community-acquired pneumonia (CAP) is Streptococcus pneumoniae.^1,2 Infections with Mycoplasma pneumoniae are rarer.^3-5 In Canada and the United States, macrolides are recommended as first-line antibiotics for adults with CAP.⁶ From 1993 to 1999, macrolide prescription rates increased 3-fold, particularly in children.⁷ Unfortunately, macrolides no longer cover all bacterial causes of CAP. Resistance rates of the most common pathogen isolated from patients with CAP, S pneumoniae, are dramatically rising.⁷ A recent study in Pittsburgh, Pa, schoolchildren demonstrated macrolide resistance in 48% of all group A streptococci isolates from throat cultures.⁸ In consideration of these alarming resistance patterns, the Active Bacterial Core Surveillance/Emergence Infections Program Network has urged physicians to reduce inappropriate prescription practices, particularly in young children.⁷

After confirming the diagnosis of CAP in a child, the physician must decide on antibiotic prescription and further diagnostic efforts. Although macrolides remain the antibiotic of choice in patients with M pneumoniae,⁹ β-lactams offer an alternative for other frequent bacterial infections. Rapid detection of M pneumoniae, which is now possible by means of polymerase chain reaction analysis,¹⁰ precludes inadequate prescriptions. However, application of this test to all children is costly. Moreover, while awaiting the polymerase chain reaction results, most physicians prescribe a first-line antibiotic. Risk stratification by simple clinical decision rules, which assist targeted use of costly tests and increase the chance of prescribing the most appropriate antibiotics in the first place, is highly desirable.

Time is crucial in the domain of clinical decision rules.¹¹ In general, most clinical decision rules do not provide perfect information but assist in gauging the likelihood of a disease¹¹ or in the triage of patients.^11,12 The rules incorporate key variables from patient histories, results of the physical examination, and initial laboratory evaluation.¹¹ Potentially useful rules achieve risk gradients of greater than 10 between the groups at lowest and highest risk or an area under the receiver operating characteristic curve (c statistic) of greater than 0.75.^11,12 Based on logistic regression analysis, decision rules¹³ providing algorithms for the computation of scores and disease risk¹² or, alternatively, fast-and-frugal heuristic decision trees may have derived. The latter are easier to memorize than many rules. These trees model everyday human decision making and are based on elementary, sequential inference rules.^14,15 Their underlying principle is elimination of alternative diagnoses in a few simple and successive steps. Trees start with the most diagnostic criterion at the top.¹⁶ Well-performing trees should be as effective clinically as full logistic regression models. Theoretically, the inherent advantage of fast-and-frugal trees is a robustness in settings other than the derivation data set.¹⁶ They also eliminate the need for computations or consultation of scoring tables.¹⁷

Decision rules for predicting the underlying etiology in pediatric CAP are lacking. Therefore, we developed 2 prediction rules from readily available clinical criteria for rapid risk stratification of M pneumoniae as a cause in children with CAP. The first rule was a score derived from logistic regression analysis; the second, a fast-and-frugal decision tree.

We conducted a 24-month cohort study at the emergency department and walk-in clinic of a tertiary care university hospital. Children and adolescents (aged 1 month to 16 years) with CAP confirmed (by 2 reviewers) by findings on chest x-rays were eligible. The initial laboratory workup included a differential white blood cell count and measurement of C-reactive protein level. To establish the cause of CAP, blood cultures and nasopharyngeal secretions for detection of respiratory viruses, Chlamydia pneumoniae, and M pneumoniae were obtained. Antigens from influenza A and B viruses; parainfluenza viruses 1, 2, and 3; respiratory syncytial virus; and adenoviruses were identified by means of an enzyme-linked immunosorbent assay. Antibody testing in paired serum samples (at baseline and 4-week follow-up) served as the diagnostic gold standard for acute M pneumoniae infection. Patients without follow-up samples were excluded. Further details have been reported elsewhere.¹⁰ The study was approved by the institutional ethics committee.

The starting point for the model development was a data set containing all variables from history, clinical examination, and initial laboratory and radiological workups that were available when the diagnosis of CAP was established. For development of the logistic regression model, we subjected all variables to a univariate screen for an association with the outcome (confirmed infection with M pneumoniae). We tested 2-variable combinations to rule out negative confounding. Using a stepwise inclusion procedure, we identified a simple model with only few variables. We transformed the regression coefficients to derive a scoring system by means of a 2-step procedure. First, we multiplied and rounded the regression coefficients to achieve full numbers. Next, we tested further simplifications to arrive at a scoring system with a sum of 10 points. Of the models that did not significantly differ according to the Akaike information criterion, we selected the model with the best fit (checked by means of the Hosmer-Lemeshow test¹³) and a simple scoring table.

For development of the fast-and-frugal decision tree, first, we identified the variable with the highest sensitivity for M pneumoniae. In this case, a negative result assists in ruling out M pneumoniae infection. In fast-and-frugal trees, the most discriminative variable is placed at the top of the tree.¹⁵ The simplicity of the trees arises from the specific choice of variables and cutoffs; these allow physicians to rule out (or to rule in) a particular disease at each decision knot for a large proportion of patients by deliberately accepting a small false-negative (or false-positive) rate.

We used 2 criteria to compare the performance of the decision rules. First, we compared the c statistics (mathematically equivalent to the area under the receiver operating characteristic curve) of the logistic regression model, the score, and the fast-and-frugal tree. We used the following 2 methods to compare the performance of the models: (1) the Akaike information criterion,¹³ and (2) the algorithm suggested by Hanley and McNeil.¹⁸ Second, we determined the rate at which each rule would allow reduction of macrolide prescriptions compared with a strategy of giving macrolides to all patients with CAP. This comparison was performed for the same false-negative rate in both models. In the absence of a validation data set, we derived confidence interval (CI) estimates by the observed distribution of c statistics from 1000 bootstrap cycles. We used SAS software (Version 8.1; SAS Institute Inc, Cary, NC) for the analyses.

Informed consent was obtained for 323 of 472 children with confirmed CAP. Paired serum samples were available in 253 patients. Seventy children failed to attend the follow-up visit (data not shown). However, their baseline characteristics did not differ from those of the patients included. Acute infection with M pneumoniae was serologically proved in 32 children (13%). The remaining 221 cases were classified as due to C pneumoniae (n = 1), positive bacterial blood cultures (n = 13), positive respiratory viral antigen test (n = 40), and mixed or unknown causes (n = 167).

Compared with other children with CAP, patients with M pneumoniae were older, had a longer duration of fever, lower leukocyte levels, and lower absolute neutrophil counts. No other variable retained a significant univariate association (Table 1).

Multivariable logistic regression analysis identified a model consisting of age and the logarithm of duration of fever (c = 0.84; Hosmer-Lemeshow test, P = .29). Table 2 and Table 3 present the scoring system derived from the regression coefficients. The score yielded c statistics of 0.84 (observed 95% CI, 0.77-0.91). The system stratified children into groups at low (absolute risk [AR], 2%), moderate (AR, 7%), high (AR, 28%), and very high (AR, 65%) risk for M pneumoniae. Using the strata instead of the full score resulted in a c statistic of 0.84 (95% CI, 0.77-0.90). Based on the Akaike information criterion, the full model performed significantly better than the strata model (χ² test, P<.05).

Figure 1 shows the fast-and-frugal decision tree. The first question on the history of fever eliminated 95 (38%) of 253 patients, at the cost of missing 3 patients with M pneumoniae infection (sensitivity, 90%; 95% CI, 75%-98%). The second question (age ≤3 years) eliminated an additional 85 patients (34%) at the cost of missing 6 patients with M pneumoniae infection (sensitivity, 79%; 95% CI, 60%-92%). In the remaining 73 patients (29%), the AR for M pneumoniae infection was 32%. The c statistic of the tree was 0.76 (95% CI, 0.70-0.83). According to the Akaike information criterion, the score model discriminated significantly better than the fast-and-frugal tree (P = .008). However, using the more conservative algorithm suggested by Hanley and McNeil¹⁸ and considering the correlation between the 2 systems (r = 0.63), the difference of the areas under the curve was not significant (2-sided P>.20).

To assess the agreement between the scoring system and the fast-and-frugal tree, we collapsed the high- and very-high-risk categories from the scoring system to a single category. The weighted κ, which indicates the agreement beyond chance of the 2 classification systems, was 0.68. Most of the disagreement between the systems occurred at the discrimination between low and moderate risk (Table 4).

Our objective was to compare a clinical decision rule derived from logistic regression analysis and a fast-and-frugal decision tree for risk stratification of M pneumoniae infection in children with CAP. Simple data gathered while establishing the diagnosis of CAP (duration of fever and patient age) identified patients at low, medium, high, and very high risk. The summary score yielded a gradient of greater than 30 between the low- and the very-high-risk groups. Compared with the score, which requires looking up data in a table, the fast-and-frugal decision tree can easily be memorized. Asking the simple question regarding duration of fever and the age of the child allows identification of the following group at high risk (32%) for CAP due to M pneumoniae: children with CAP who have had fever for more than 2 days and who are older than 3 years. The score model places 75% of all patients with M pneumoniae infection into the high- or very-high-risk group. The fast-and-frugal tree achieves a marginally lower correct classification rate (sensitivity, 72%; Table 2 and Table 3 and Figure 1). The agreement between both systems was moderate (κ = 0.68), with most of the disagreement occurring at the distinction between low and moderate risk. This indicates that the 2 systems represent 2 distinct approaches that best agree at identifying patients at high risk for M pneumoniae infection.

When the external validity of the rules is established in an independent patient population, would either of these rules have any clinical usefulness? They do not provide perfect information, which is counterbalanced by the benign natural course of CAP due to M pneumoniae. In otherwise healthy patients, M pneumoniae infection rarely proceeds to a life-threatening disease. More often, the disease resolves spontaneously. The probability of a dramatic progression of the disease during the next 48 hours in patients infected with M pneumoniae should be considered when applying the rule, because up to 1 in 4 patients will be missed if initial prescriptions are based on the score alone. This false-negative rate would be unacceptable for patients at high risk for adverse outcomes. Physicians in the United States who prescribe macrolides to all children with CAP who are younger than 5 years face a resistance or decreased susceptibility rate of 30%⁷ or, according to a recent survey, even 48%.⁸ Applying the decision rules would mean that 25% of all patients with CAP due to M pneumoniae will not receive macrolides. Because CAP due to S pneumoniae is more common than CAP due to M pneumoniae,³ physicians applying either of the decision rules have a better chance of prescribing effective antibiotics in the first place. Basing decisions about first-line antibiotic treatment on the rules would reduce macrolide prescription rates by 75% when using the scoring system and by 68% when using the fast-and-frugal tree rule. These reductions may be rewarded by declining resistance rates of S pneumoniae.¹⁹ An independent issue is the appropriateness of treating infections due to M pneumoniae with macrolides. Future studies will have to demonstrate that prescribing macrolides to children with CAP due to M pneumoniae is associated with improved outcomes.

An alternative strategy for curtailing inappropriate macrolide prescriptions is to establish the underlying cause, in particular M pneumoniae, in every case. At present, applying a rapid molecular diagnostic test for M pneumoniae in all children with CAP would incur diagnostic costs (in US dollars) of $70 per patient undergoing testing. An analysis of the costs per additionally identified case of M pneumoniae (marginal cost comparison) shows that the decision rules may also aid cost-conscious decision making about further tests. Application of the scoring system results in estimated costs per identified case in the very-high-risk group of $108, whereas extension of testing to the high-risk group results in marginal costs of $248 per detected case. Addition of the medium-risk group incurs marginal costs of $1061, and inclusion of the low-risk group escalates the costs to $3465 per additionally identified case. The corresponding figures using the fast-and-frugal tree are $190 for each case detected when children older than 3 years who have had fever for more than 2 days undergo testing. The marginal costs rise to $883 if only duration of fever is used, and to $2216 if testing is extended to all. Considering the clinical consequences of missing M pneumoniae infection, molecular testing in the low-risk groups is probably not cost-effective.

We acknowledge the possibility of recall bias regarding the duration of fever. However, it is unlikely that recall bias would have differed across the causes. Because of the small number of true-positive cases, we were unable to divide the data into a derivation and a validation data set, and we were forced to test the robustness of the models by bootstrap simulation. However, the CIs remain large in the very-high-risk group of the scoring system. Therefore, the presented scoring system and the decision tree should be validated in an independent patient sample. They may not be applicable during an epidemic of M pneumoniae.

We demonstrated that a scoring system and a fast-and-frugal decision tree provide a rapid probability estimate of the cause of childhood CAP. These simple rules may aid physicians in cost-conscious and efficient ordering of costly diagnostic tests and increase the chance of prescribing appropriate first-line antibiotics. The fast-and-frugal decision tree suggests that first-line macrolide therapy may be restricted to children with CAP who have had fever for more than 2 days and who are older than 3 years.

Accepted for publication May 20, 2002.

This study was supported in part by Pfizer AG, Zurich, Switzerland.

Liberal prescription of macrolides in children with CAP is associated with increasing prevalence of resistant pathogens. Reserving macrolide prescription to patients infected with Mycoplasma pneumoniae is an option to curb excessive use. Simple decision rules are desirable to assist identifying patients at high risk for M pneumoniae infection.

The study presents 2 simple prediction rules that allow stratification of children with CAP into groups at low, medium, and high risk for M pneumoniae as a causative agent.

Corresponding author and reprints: David Nadal, MD, Division of Infectious Diseases, University Children's Hospital of Zurich, Steinwiesstrasse 75, CH-8032 Zurich, Switzerland (e-mail: david.nadal@kispi.unizh.ch).