A likelihood ratio nomogram. Assuming a pretest probability of 5%, the likelihood ratios for a cerebrospinal fluid white blood cell count of greater than and less than 30/µL result in posttest probabilities of 35% and 1.4%, respectively.
Frohna JG, Park SM, Gopal S. Diagnosing Bacterial Meningitis After the Haemophilus influenzae VaccineStill A Challenge. Arch Pediatr Adolesc Med. 2001;155(12):1307-1310. doi:10.1001/archpedi.155.12.1307
DIMITRI A.CHRISTAKISMD, MPHHAROLD P.LEHMANNMD, PhD
THIS STUDY1 explored the usefulness of various characteristics of cerebrospinal fluid (CSF) in making a diagnosis of bacterial meningitis, looking specifically at a period during which most children have been immunized against Haemophilus influenzae type b. In this retrospective study, the authors analyzed the CSF examination results from 1617 children who underwent a lumbar puncture to diagnose community-acquired meningitis. There were 44 cases of definite or presumed meningitis. Of these, 11 had fewer than 30 white blood cells (WBCs) per microliter. Using this value as a cutoff, they calculate likelihood ratios (LRs) for a positive and negative test of 10.3 and 0.27, respectively. They also found that other significant predictors of bacterial meningitis were age, CSF glucose, CSF protein, Gram stain, CSF–serum glucose ratio, and peripheral blood band count. They conclude that children older than 6 months with a CSF WBC count of less than 30/µL (without other abnormal CSF findings) are at low risk for bacterial meningitis.
We evaluated this study in terms of its validity, importance, and applicability using the standard criteria for evaluation of an article on a diagnostic test.2- 4 This was done to determine the quality of the evidence, the utility of the diagnostic test, and relative costs and benefits.
The gold standard was bacterial culture or, for those who had received prior treatment with antibiotics, latex agglutination. Cases were classified as presumed bacterial meningitis when the diagnosis was not made by culture and if the patients were treated as if they had meningitis (based on clinical signs and symptoms). This is a conservative measure, and reasonable given the seriousness of the diagnosis.
Ideally, a gold standard should be applied in a manner that is independent of and blinded to the results of the diagnostic test. While this study does not describe this aspect of the methods, it is less of a concern because both the CSF WBC count and the results of the CSF culture are relatively objective measures, into which it would be arguably difficult to introduce bias.
The types of patients in the study can be assessed by examining the inclusion and exclusion criteria. While this study was performed in a tertiary care hospital, and thus may not be reflective of patients seen in general practice, the authors selected subjects for inclusion to identify those who were being evaluated for community-acquired meningitis. Thus, they excluded (1) clotted samples; (2) samples with a CSF red blood cell count higher than 10 000/µL; (3) ventriculoperitoneal shunt samples; (4) repeat lumbar punctures performed within 14 days; (5) any underlying condition that could predispose the child to bacterial meningitis; and (6) patients younger than 2 months or older than 17 years. While these exclusions are sensible, it is important to note that these criteria resulted in the exclusion of 9837 samples (86%). This may raise concern regarding the validity of the study populations, but review of Table 1 of their article indicates that the largest proportion of these samples were excluded because they were not obtained on selected units or were from patients younger than 2 months or older than 17 years. The authors eliminated specimens obtained from some hospital units because their hospital cares for a large number of neonatology, neurosurgery, and oncology patients who routinely had CSF samples sent for a number of complex illnesses (P. Dick, MD, personal communication, September 14, 2001), and the goal was to focus on the use of this diagnostic test in the primary care setting.
The study was also done retrospectively, and it is possible that the patients who were included in this study were somehow different or more likely to have a CSF pleocytosis than if they had been included in a prospective manner.
All patients had a CSF culture obtained or latex agglutination performed regardless of test results. This is important so as to avoid misclassification of patients as having or not having a disease, based on diagnostic test results alone. For example, if the authors had not performed CSF culture on all children with a CSF WBC count lower than 30/µL, they would have missed 11 children with definite or presumed bacterial meningitis.
An LR for a given test result is defined as the likelihood of having that test result in the presence of disease, compared with the likelihood of having that same test result in the absence of disease. The LR is the most useful way to express the importance of a diagnostic test for 3 reasons: (1) it combines the properties of both sensitivity and specificity into a single number, and thus, reflects characteristics of the test and is independent of the disease prevalence; (2) LRs can be calculated for multiple levels of the test; and (3) an LR is used to translate a pretest probability into a posttest probability. An LR of 1 results in no change in the posttest probability. An LR greater than 1 increases the posttest probability; whereas an LR less than 1 decreases it. Because of this, the further the LR from 1, the more helpful the test. A general rule of thumb is that LRs greater than 8 to 10 and less than 0.1 are most useful (ie, the resultant posttest probabilities will be significantly different than the pretest probabilities).
This article includes data necessary for the calculation of multilevel LRs for the CSF WBC count (Table 1). As an example, for a CSF WBC count of 0/µL to 3/µL, we can calculate the LR as (5/44)/(1096/1573) or 0.16. This means that a child is 0.16 times as likely to have a CSF WBC count of 0/µL to 3/µL with bacterial meningitis as to have this same CSF WBC count and not have bacterial meningitis. Likelihood ratios for each of the other ranges of CSF WBC counts specified by the authors are 0.59, 4.9, and 27.8, respectively. When the authors chose to look just at those children whose CSF WBC counts were greater than or less than 30/µL, they converted the multilevel test into a dichotomous one. This would yield a standard 2 × 2 table, with a "positive" test defined as a CSF WBC count greater than 30/µL. Using this cutoff point, one obtains LRs of 10.3 for CSF WBC counts greater than 30/µL and 0.27 for WBC counts less than 30/µL. As described above, these values for the LRs are potentially quite useful in modifying the pretest probability.
Likelihood ratios are more helpful than positive predictive values and negative predictive values since the latter 2 depend on the prevalence of the disease. One can instead use the same LR for almost any pretest probability and easily compute a posttest probability using a standard LR nomogram. As an example, let's assume a pretest probability of bacterial meningitis of 5% based on clinical signs and symptoms. Using the above LRs, we can determine posttest probabilities as shown in Figure 1.
The authors don't provide the data needed to calculate the LRs for different ages, but they do indicate that this was an independent predictor for bacterial meningitis in the study population. Because the ability to combine LRs from independent tests is another useful feature of LRs, we contacted the authors for these data. Using a cutoff of 12 months, the authors obtained LRs of 1.4 (95% confidence interval [CI], 1.0-1.8) for children between 2 and 12 months of age and 0.7 (95% CI, 0.5-1.0) for those older than 12 months (P. Dick, personal communication, September 14, 2001). To combine age with CSF WBC count, multiply the relevant LRs together. For example, a 9-month-old child with a CSF WBC count of 31/µL to 300/µL would have a combined LR of 4.88 × 1.4, or 6.83.
Finally, the authors calculate a sensitivity and a specificity of 100% and 47.7%, respectively, using an algorithm that calls for hospitalization and presumptive treatment of children who fall into any of the following groups: (1) CSF WBC count greater than 30/µL; (2) younger than 6 months; or (3) abnormalities in CSF glucose, protein, Gram stain, serum–CSF glucose ratio, or peripheral blood band count. This results in a positive LR of only 1.9 (one would admit and begin treatment for many children without bacterial meningitis), but a negative LR of 0 (one would not miss any children with bacterial meningitis in this study).
This diagnostic test is readily available and affordable. Since CSF analysis is a standard diagnostic test, it should perform similarly in our setting.
We can generate an estimate of our patient's pretest probability through prevalence statistics, clinical experience, or from the study itself. Prevalence data on bacterial meningitis are difficult to find. Most data are from the pre-Haemophilus vaccine era. We do know that Haemophilus meningitis has decreased by 95% since the introduction of the vaccine,5 so we can assume the pretest probability of bacterial meningitis is low, but how low is it? Clinical experience may cause clinicians to significantly vary their estimate of the pretest probability. For example, we did an informal poll of faculty and residents in general pediatrics, infectious diseases, and emergency medicine at our institution, asking what the probability of bacterial meningitis was in a 6-month-old patient with a temperature of 102°F and clinical symptoms that were felt to warrant a lumbar puncture. Among the residents, approximately half of the sample (n = 33) indicated that risk was in the 3% to 5% range, whereas among faculty, the risk estimate was lower, with 40% of the respondents (n = 30) indicating a risk of less than 1%. Using the prevalence in this study would yield a pretest probability of 2.7%, which may be high (especially considering that we are now immunizing against Streptococcus pneumoniae), but it is a reasonable starting point.
Does the result move us across a test-treatment threshold? Just what should this threshold be? In other words, at what posttest probability would we as clinicians be comfortable that the child truly has or does not have bacterial meningitis, a potentially life-threatening condition?
When we asked our faculty and residents this question, we again had divergent answers. The residents were more risk averse, with 78% of respondents wanting the posttest probability to be less than 0.1%. In contrast, more than 50% of the faculty were willing to accept a risk of 1 in 500 or greater. While clinicians do not usually think in terms of probabilities, we each generate qualitative risk thresholds every day on which we base our clinical decisions.
Assuming a pretest probability of 2.7%, a CSF WBC count of 30/µL would mean that the patient has a 22.2% chance of truly having bacterial meningitis, whereas a CSF WBC count of less than 30/µL would mean that the patient still has a 0.7% chance of having bacterial meningitis. Since bacterial meningitis is such a serious disease, is a 7 in 1000 chance low enough to send the child home? Our faculty would say yes, while the residents would say no.
Using the algorithm proposed by the authors, the LRs of 1.9 and 0 yield a posttest probability of 5.0% for having bacterial meningitis if the child falls into one of the high-risk categories, but 0% for having bacterial meningitis if not. In this case, 95% of those suspected of having bacterial meningitis will have been treated "unnecessarily" to avoid missing one child who actually has disease. Is this reasonable?
In the current medical environment, we must consider not only the clinical utility of a test, but also how the use of such a test may affect the cost and efficiency of care. Meningitis is indeed a serious disease, but the risk of missing a diagnosis can theoretically never be reduced to zero. Can we find a risk threshold through the use of these or other diagnostic studies, that would be acceptable as "low enough," yet could help avoid unnecessary hospitalization and propagation of bacterial resistance? The authors' recommendations regrading which children deserve inpatient management seem reasonable keeping in mind that a fairly large number of children will be admitted and will ultimately be found not to have bacterial meningitis.
The goal of this valid study was to determine if the CSF WBC count can be used to stratify children with suspected meningitis into high- and low-risk groups. Indeed, it does seem that a child older than 6 months with a CSF WBC count of less than 30/µL, without other abnormalities of the CSF, is at a significantly lower risk of having bacterial meningitis. However, as with all clinical decision-making, the ultimate use of these diagnostic tests will depend on the clinician's willingness to accept risk.