Can a Clinical Decision Rule Decrease Antibiotic Use in Viral Meningitis?

IN THIS issue of the ARCHIVES, Oostenbrink et al^1,2 revisit a prior study of children with meningeal signs who came to the emergency department at Sophia Children's Hospital in Rotterdam, the Netherlands, between 1988 and 1998. In the earlier study,² the authors developed a prediction rule that identified 35% of patients with meningeal signs who did not require lumbar puncture. The study described in this issue seeks to extend that rule by incorporating cerebrospinal fluid (CSF) results with clinical findings to identify children from whom empiric antibiotic therapy can safely be withheld. It combines both a training set and a validation set from the earlier study to use as a new training set. The final rule is a clinical/blood laboratory test score detailed in the article's "Methods" section combined with a CSF variable score best described in its footnote to Table 3 ; the treatment category can be found in Figure 1.¹

Using standardized criteria,^3-5 we evaluated this study first in terms of its validity as a diagnostic test and then for its utility as a clinical decision rule. This was done to determine the quality of evidence, value of the clinical decision rule, and applicability of the rule in common clinical practice.

The gold standard for the diagnosis of bacterial meningitis is the detection of bacterial growth in a CSF culture, which was fulfilled by most cases of bacterial meningitis in the study. Three patients had mild pleocytosis in conjunction with a blood culture positive for Neisseria meningitidis, and 2 had CSF pleocytosis with no blood or CSF abnormalities but a CSF culture that was positive for N meningitidis antigen. Both of these latter examples seem reasonable to include as cases of bacterial meningitis. However, the inclusion of 7 children among the 99 cases of bacterial meningitis in the study population is questionable. Five children with cultures negative for N meningitidis had received oral antibiotic therapy prior to lumbar puncture and were treated with antibiotics for at least 7 days; they could easily have been misclassified. An additional 2 children were included as bacterial meningitis cases on the basis of nose or throat cultures that were positive for N meningitidis. Because nasopharyngeal carriage of this organism is relatively common,⁶ this latter criterion is not a reliable indicator of meningitis etiology. Therefore, 7% of bacterial meningitis cases in this study population were not subjected to a well-accepted reference standard. Aside from these cases, it does appear that the test results (clinical and CSF findings) were assessed independently of the reference standards of blood and CSF cultures and CSF antigen detection.

The study group may not reflect all patients seen for whom meningitis was part of the differential diagnosis. The population described seems somewhat different from that in most studies of meningitis in children. Overall, 44% of the study population was determined to have bacterial meningitis, and 36% had a disturbed level of consciousness, defined as being unresponsive or responsive to pain only.² The authors note that the patient eligibility criteria consisted of coding by emergency department pediatricians as having meningeal signs, rather than other clinical indicators of meningitis. It is conceivable that this classification system might select patients with a more severe form of the disease. Neither of the authors' studies mentioned the number of patients with meningitis at their institution who were not included.

The only situation in which this issue arose concerned implied bacterial meningitis. 2 patients had CSF pleocytosis and mucosal swabs positive for N meningitidis. It is unclear if these patients underwent the swabs before or after the CSF cultures were interpreted. If these tests were performed after the CSF cultures were determined to be negative for N meningitidis, the decision to do the mucosal cultures would have been based on the negative reference standard.

Although the methods of CSF analysis and blood and CSF culture techniques were not described in the article, these techniques are relatively standardized and easily replicated in clinical laboratories. The clinical parameters used in the original prediction rule were well described in the prior article.²

Our article includes the data necessary for the calculation of multilevel likelihood ratios for the scores derived from the CSF scoring model in the defined ranges of 2 to 4, −1 to 1.5, −2.5 to −1.5, and −5 to −3 (Table 1). Posttest probabilities can be calculated using the Fagan nomogram.⁴ Unfortunately, the CSF and clinical risk scores are not provided for each of the boxes in the authors' Figure 1, so likelihood ratios cannot be calculated from these combined data.

Reproducibility can be limited by problems with the test as well as its interpretation. For this study, serum C-reactive protein (CRP) results had to be available on an emergency basis, which is not the practice in all institutions. The article reports that analyses supported the application of the CSF rule both in patients undergoing lumbar puncture as selected by the clinical rule and in those undergoing the procedure based on the pediatrician's decision. These analyses potentially address this limitation; however, specific data were not provided.

Additionally, reproducibility may be further limited by the interpretation of CSF indexes in the case of the traumatic lumbar puncture. This circumstance was not discussed, and it is unclear if the application of the CSF rule was (or should be) altered in these cases.

Applicability to our patient population may be limited by several factors. First, the population in this study may represent a group with more severe illness than most children with meningitis, as stated previously. Second, although a demographic profile of the patient population served by Sophia Children's Hospital is not provided, there are likely to be considerable differences between the populations of Rotterdam, the Netherlands, and Washington, DC, in terms of ethnicity, education level, epidemiological factors related to meningitis, and other issues. In addition, the study occurred during the prevaccination period in which Haemophilus influenzae type B (Hib) was more frequently a cause of meningitis. This factor could affect the use of the rule in any country with universal Hib vaccination. The authors acknowledge this fact and state that modification of the results to exclude patients with Hib yielded the same CSF and clinical-scoring rule. Furthermore, widespread use of conjugate pneumococcal vaccine may limit the applicability of these results to future practice if this vaccine significantly affects epidemiological factors regarding bacterial meningitis, as has been predicted.⁷ Other causes of meningitis, such as Lyme disease, apparently were not considered but are relatively common in our population. Finally, the study examined patients aged 1 month to 15 years. Many pediatricians support obtaining a lumbar puncture in infants up to age 3 months for the evaluation of fever, bypassing the application of the initial clinical rule in this age group. More information is needed regarding the use of this rule in very young infants.

When deriving a clinical decision rule, investigators generally construct a list of potential predictors of the final outcome based on aspects of the history, physical examination, and laboratory tests.⁵ The authors of this study relied on a list of predictors generated from a previous study.² From these predictors, they produced a clinical decision rule and risk score and then selected those patients at higher risk for bacterial meningitis to be included in the new study. We reviewed the initial study to determine whether the authors included all of the important predictors in the derivation process for the clinical decision rule.

From the initial study, the investigators determined that the most significant predictors of bacterial meningitis were duration of illness, the presence of vomiting, meningeal irritation, cyanosis, petechiae, disturbed consciousness, and an elevated CRP level.² These factors were used in the new study to generate the second clinical decision rule.

Most clinical features that may appear in a patient suspected of having bacterial meningitis on the basis of meningeal signs were considered in the initial study. The authors' definition of meningeal signs, however, included neck pain; the cause of this sign can be highly variable, and its presence on a physical examination may depend on the skill of the examiner. One wonders whether obvious cases of cervical adenitis, for example, were included in the original study population. In addition, the authors did not consider patients without meningeal signs in their derivation, such as those with coma or seizures who might also warrant an evaluation for bacterial meningitis.

In deriving the clinical decision rule for the new study, the authors focused on the initial CSF findings.¹ These indexes included the CSF leukocyte count, percentage and absolute number of polymorphonuclear leukocytes, CSF protein and glucose concentration, CSF/blood-glucose ratio, and CSF gram stain, all of which are commonly performed on CSF samples. Because other investigators have commented on the diagnostic utility of CSF CRP determinations,^8-10 it might have been useful to evaluate CSF CRP in the derivation of the rule.

Because this study relied on information obtained retrospectively, the authors report that not all data were available for each patient. In some cases, imputation techniques were used to fill in missing variables. However, patients for whom more than 50% of the data were missing were excluded from the analysis. Imputation is a statistical technique that fills in missing data by inference using data from the patient population as a whole. It is not commonly used in medicine, and it would have been helpful to know how many data elements were truly missing from the data set. In regard to the laboratory predictors, 16 of 227 patients included in this analysis did not receive lumbar puncture. It would then follow that in the 16 cases without a lumbar puncture, data on the preliminary CSF indexes may have been derived using imputation. Although the authors used credible imputation methods in this study, we are still concerned that the validity of the study is lessened by the use of imputed data.

The investigators define the outcome event to be the presence or absence of bacterial meningitis.¹ For patients who did not undergo CSF analysis, follow-up was achieved with an outpatient department visit or telephone call. Although not specifically mentioned, the authors imply that 100% follow-up was accomplished.

The investigators in this case do not specifically address this question. Data collection appears to have been conducted retrospectively, identifying patients using the problem-oriented patient classification system at the institution. A pediatrician coded each patient's initial complaint, and the remainder of the medical record was completed in a standardized fashion with respect to history of the main problem, complaints regarding major organ systems, physical examination findings, and outcome diagnosis.² Laboratory data for this study were obtained from a computer-documented hospital information system. Because of the retrospective nature of the investigation, it is not possible to know whether the physician coding the chief complaint had any knowledge of the CSF findings or the final diagnosis. Knowledge of the initial predictors and/or final outcome could potentially result in bias when completing the medical record.

It is difficult to determine how large a sample size should be to derive a clinical decision rule. This study was conducted during a 10-year period between 1988 and 1998 in Rotterdam, the Netherlands. It included 360 patients whose chief complaint involved meningeal signs; it then evaluated 227 patients who were judged to be at significant risk for bacterial meningitis. Wasson et al¹¹ suggested that the smallest prognostic or diagnostic category in a training set should contain at least 5 patients for every predictive finding in the rule. Table 2 in the earlier study lists 30 variables examined by univariate analysis, and 7 CSF-related variables were added in the new study. The final rule includes 9 variables (7 clinical and 2 CSF values), which according to the rule of Wasson and colleagues would require 45 patients with bacterial meningitis to be included in the training set. With a total of 92 patients with bacterial meningitis (excluding the questionably categorized patients), this criterion was achieved.

For a clinical decision score to predict the likelihood of bacterial meningitis, the rule should include both clinical and laboratory variables, as this one does. The rule is biologically plausible.

The earlier study was validated prospectively in a small group of children at the same institution.² In the new study, the authors used random bootstrap techniques as an internal validation tool.¹ Although this information is useful, internal validation techniques do not address the possibility that the predictors identified in this study are particular to either the specific patient population in the Netherlands, the point in time it was measured, or the physicians involved in the study. Prospective validation is needed, preferably with a large and different patient population.

Concerns regarding the patient selection criteria, retrospective nature of data collection, possibilities of nonequivalent patient populations due to both epidemiological and vaccine-related considerations, and use of imputation to fill in missing data will not allow us to use this rule in our current practice.

In settings where all patients with meningeal signs undergo lumbar puncture and all patients with abnormal CSF findings are treated with empiric antibiotics, this rule could be helpful. According to the evidence provided in the authors' 2 articles, the rule could be used in a comparable clinical setting with the appropriate laboratory support to lessen the use of empiric antibiotics in patients with meningeal signs, particularly in those determined to have an intermediate clinical risk or CSF score. The duration and costs of hospitalization would also be reduced.

The goal of this study was to derive a clinical decision rule, including CSF indexes and clinical characteristics, that would aid the physician in the decision to treat patients with initial meningeal signs with empiric antibiotics while awaiting the results of bacterial cultures. For a clinical decision rule to be effective, a change in physicians' behavior regarding the application of the rule needs to be demonstrated, as well as an improvement in patient outcome. Despite various limitations in the design of the study, the authors have derived a rule that makes clinical sense and could potentially be useful during the initial evaluation of patients suspected of having bacterial meningitis. However, we would place this decision rule at level 4 because it was derived and validated only in a retrospective database using statistical techniques.⁵ To achieve a rule that can be used in many different settings with a high probability of improving outcome (level 1 evidence), we suggest that the rule should next be validated in at least 1 large, broad-based prospective study and 1 impact analysis study.

Corresponding author: Bernhard L. Wiedermann, MD, Pediatric Residency Training Program, Children's National Medical Center, Department of Pediatrics, The George Washington University School of Medicine and Health Sciences, 111 Michigan Ave NW, Washington, DC 20010 (e-mail: bwiederm@cnmc.org).