Copyright 2000 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.2000
To describe parents' values for outcomes of occult bacteremia using utility assessment, a quantitative method that incorporates risk preference.
Computer-based utility assessment interview.
Urban children's hospital pediatric emergency department with 50,000 visits annually.
Convenience sample of parents presenting with a child between 3 and 36 months.
Main Outcome Measure
Parents' utility values for 8 outcomes from treatment of occult bacteremia: blood drawing, localized infection, hospitalization for antibiotics, meningitis with recovery, meningitis resulting in deafness, minor brain damage, severe brain damage, and death.
Ninety-four subjects successfully completed the interview. Mean utilities were 0.9974 for blood drawing, 0.9941 for local infection, 0.9921 for hospitalization, 0.9768 for meningitis with recovery, 0.8611 for deafness, 0.7393 for minor brain damage, 0.3903 for severe brain damage, and 0.0177 for death. All values were significantly different from those that immediately preceded and succeeded (P<.0001), except for local infection vs hospitalization (P = .14). Median utilities for blood drawn, local infection, and hospitalization were 1. There were no significant differences among utilities of parents who presented with a febrile child (temperature ≥39°C), or an afebrile child (temperature <39°C). There were also no significant differences among utilities regardless of whether parents had children with prior experience with the outcomes.
Assessment of utilities for outcomes of occult bacteremia yielded extremely high mean and median values for outcomes without permanent sequelae. This suggests that parents presenting to an emergency department may rationally prefer painful transient experiences, including venipuncture, for their children rather than risk even rare chances of severe outcomes.
THE TREATMENT of the young febrile child without obvious focal infection remains controversial. Recent studies document a 3% risk of occult pneumococcal bacteremia in children aged 3 to 36 months with a temperature higher than 39°C.1 With the virtual elimination of Haemophilus influenzae type b as an invasive disease,2 the diagnosis and treatment of occult bacteremia now focuses primarily on Streptococcus pneumoniae, which has a high spontaneous resolution rate and less frequently results in invasive disease, most importantly meningitis. However, pneumococcal bacteremia is not risk-free. The probability of meningitis after pneumococcal bacteremia has been estimated to range from 2.7% to 5.8% in 2 recent meta-analyses.3,4 As only a small percentage of children suffer severe adverse sequelae from pneumococcal bacteremia, it is not surprising that there is considerable practice variation among pediatricians evaluating children at risk for bacteremia.5 Expert opinion is equally diverse, ranging from those who recommend "no testing other than urine culture"6 to those who believe that "screening white blood cell count and expectant antibiotic therapy remain the best choice for some febrile children."7 The diversity in recommendations may reflect variations in values for outcomes. If the treatment decision hinges on values, it is particularly important to understand parents' values for the outcomes.
Utilities are an established method for quantifying preference for a specified health state. The standard gamble is a method for measuring utilities that is founded in the axioms of decision theory, incorporates the element of risk preference in the measurement,8 and has been used frequently in health care research. Prior studies employing other methods have suggested that parents may prefer treatment strategies that increase the risk of rare poor outcomes to avoid immediate short-term painful experiences, such as venipuncture.9,10 Our goal in this study was to describe parents' values for outcomes of occult bacteremia and its treatment by measuring their utilities for these outcomes.
We interviewed a convenience sample of parents presenting to an urban children's hospital emergency department. Parents were eligible for interview if the child for whom they were seeking care was 3 to 36 months old or if there was a sibling 3 to 36 months old. Eligible subjects were subclassified as to whether their children had temperatures of 39°C and higher at the time of the interview. Subjects were excluded if they could not read English or if their children were classified as critically ill at triage. The primary sample size estimate was made by calculating the number of subjects needed to achieve a 95% confidence interval of ±0.05 around utilities, with an estimated SD of 0.2. This would have required 62 subjects.11 As we were also interested in examining whether parents with febrile children who potentially faced the problem of occult bacteremia had different utilities from parents with afebrile children, we calculated the number of subjects to detect a 0.1 difference in utilities (estimated SD, 0.2) between febrile and afebrile parent groups with 80% power at α = .05. This utility difference of 0.1 is suggested as a meaningful difference in utilities between groups.12 This required 67 subjects in each group.13 Interim analysis indicated that differences in utilities between groups were much smaller than 0.1. Because of this, enrollment was halted after 118 subjects.
Subjects were interviewed using a computer-based utility assessment tool, U-Titer II©.14 U-Titer has been used in a wide range of studies investigating utilities for questions regarding treatment of various medical conditions, including stroke prophylaxis in atrial fibrillation,15 angina,16 and psoriasis.17 After giving informed consent, the subjects interacted directly with the computer, with the primary investigator (J.E.B.) present to answer questions or provide assistance.
The order of the interview was standardized. Subjects first entered demographic data. They next read a description of the 8 outcomes of occult bacteremia and provided information regarding their children's experience with these outcomes. Subjects were introduced to a method of ordering outcomes, from best to worst, using a visual analog scale (VAS). They practiced by ranking normal vision, blindness in one eye, and blindness in both eyes from best to worst. They then ranked the bacteremia outcomes from best to worst using a second VAS. Finally, subjects were introduced to the standard gamble, the method used to measure utilities. The typical standard gamble asks subjects to choose between an intermediate sure outcome and a gamble between 2 unsure outcomes, typically the standards of perfect health with a utility defined as 1, and death with a utility defined as 0. The probabilities of the 2 unsure outcomes are varied until the subject is indifferent between the sure outcome and the gamble of 2 unsure outcomes. The probability of the sure outcome at this point is the utility. For example, in practice gambles subjects were asked to evaluate blindness. The subject was presented with a choice of being blind or choosing a gamble that has some probability of restoring normal vision but also some probability of immediate death. The probabilities are varied until the subject is indifferent between blindness and the gamble. A person who is indifferent between blindness and a gamble with a 75% chance of cure and a 25% chance of death has a utility of 0.75 for blindness. After completing practice gambles for blindness in one eye and blindness in both eyes, the subjects provided utilities for bacteremia outcomes in a series of standard gambles based on their rank order from the VAS.
It is often difficult for subjects repeatedly to weigh intermediate outcomes against gambles that include death, particularly when the intermediate outcomes may have utilities near 1.12 To avoid this problem, we used a variation of the standard gamble involving a chain of gambles based on each subjects' ranking of the outcomes. In this chain, each outcome is evaluated against a gamble of wellness and the next worst outcome in the ranking. For example, consider a scenario in which 3 outcomes are ranked between perfect health and death. Outcome 1 is evaluated against a probability of perfect health and a probability of outcome 2 (instead of death); outcome 2 is evaluated against a probability of perfect health and a probability of outcome 3 (instead of death); and outcome 3 is evaluated on the full scale against a probability of perfect health and death. As the utilities of perfect health and death are defined as 1 and 0, respectively, the other outcomes utilities can be calculated (Figure 1). The goal is to allow subjects to assess outcomes, if possible, within the central 80% of the probability scale, where responses are most accurate.12 The chain procedure attempts to avoid gambles in which individuals have very high (>0.90) or very low (<0.10) utilities for outcomes. For each individual gamble the subject was limited to a risk of adverse outcome of 0.0005. For gambles in which individuals indicated a possible change in preference order from the original rank order, the suspect gambles were reassessed on the full scale from wellness to death.
Calculation of utilities from chained gambles. Outcomes are ranked from O1 (best) to O8 (worst).
We were interested in the effects of the chained gambles on the utility values; would these result in values similar to those produced by a traditional standard gamble? Therefore, approximately 25% of subjects were randomly selected for an additional gamble. This gamble reevaluated 1 outcome on the traditional standard gamble scale from death to wellness and allowed comparison to the same outcome's utility calculated from the chained gambles. From pilot studies the outcome ranked fifth on the VAS was typically the highest-ranked outcome, with permanent sequelae, and was therefore likely to have been most affected by the chaining procedure. (Outcomes without permanent sequelae would be difficult to assess in this manner, as pilot data indicated parents were essentially unwilling to gamble against death with these outcomes, which is the reason chaining was employed.)
The outcomes descriptions were derived from a decision analysis project investigating treatment strategies for occult bacteremia (Table 1). Outcomes without permanent sequelae (blood drawn, local infection, hospitalization for antibiotics, and meningitis with recovery) described pain or discomfort involved with procedures or conditions and length of hospitalization. Outcomes with permanent sequelae described long-term functional status effects while also indicating the need to experience the temporary painful events associated with meningitis. Each outcome description was printed in 12-point type on an individual 4 × 6-in index card. The Flesch-Kincaid grade level of the descriptions was 4.7 as measured using Microsoft Word 97 (Microsoft Corp, Redmond, Wash). This readability scale assigns a grade level based on the average number of syllables per word and words per sentence.18 These cards were available for reference throughout the interview.
We calculated the utility means, medians, and 25th and 75th percentiles for each of the 8 outcomes. Nonparametric testing was used to analyze utilities: Wilcoxon signed rank test for paired comparisons of utilities of outcomes and Mann-Whitney test for comparison of utilities between groups of parents. A χ2 or Fisher exact test was used when appropriate to analyze categorical data, and a Mann-Whitney test was used to analyze continuous data between febrile and afebrile groups.
Subjects were excluded from analysis if they assigned a utility for blindness in both eyes that was greater than 0.05 higher than their utility for blindness in a single eye in the practice gambles. This indicated that the subject might not understand the gamble technique. Subjects were also excluded if they made an error in a gamble that precluded calculation of all utilities. This involved choosing any outcome rather than a 100% chance of their child being well, another indication the gamble technique was not understood. Subjects were also excluded if a gamble or gambles implied a new rank order substantially inconsistent with the rank order of the VAS. Finally, subjects who chose to terminate the interview before completion of the gambles were excluded.
Institutional review board approval was obtained for this study, and written informed consent was obtained prior to enrollment of all subjects.
A group of 118 subjects were interviewed; 94 were included in the data analysis and 24 were excluded. Of these, 5 subjects made errors in the practice gamble, 13 made errors precluding calculation of utilities for all outcomes, and 2 made errors that indicated a change in preference order. (These 2 subjects' resulting utility values for blood drawing of 0.715 and 0.495 placed this outcome below many other outcomes, including meningitis with minor brain damage. These utilities violated assertions made in previous gambles as well as multiple assertions of their own rank order on the VAS.) Three subjects wished to discontinue the interview. One interview ended because of computer failure.
Included subjects were similar to those excluded in most aspects (Table 2), although there were more African American subjects in the excluded group compared with the included group. Although not significant, there was a trend toward lower education and income levels in the excluded subjects compared with the included subjects. There were no significant differences between the parents with febrile and afebrile children, although there was a trend toward lower income in subjects with febrile children.
The mean, median, 25th and 75th percentiles, and range of utilities are shown in Table 3. Each utility was significantly different from preceding and succeeding utilities, with P<.0001, except for utilities for local infection and hospitalization, which were not significantly different (P = .14). Median utilities for outcomes without permanent sequelae, except for meningitis with recovery, were equal to one.
Table 4 shows the mean utility values for the 8 outcomes for parents with febrile and afebrile children. There were no differences in the mean utilities of these 2 groups (P>.16 for all comparisons). We examined whether utilities for outcomes might be different if the parent had prior experience with the outcome and found no significant differences. The mean utility for hospitalization for parents with previously hospitalized children (n = 47) was 0.9981, compared with 0.9857 for parents without previously hospitalized children (P = .55). The mean utility for local infection for parents with children with a history of prior local infection (n = 25) was 0.9988, compared with 0.9824 for parents with children with no prior history (P = .85). The mean utility of venipuncture for parents whose children had previously undergone venipuncture (n = 75) was 0.9965, compared with 0.9997 for those parents whose children had not previously undergone venipuncture (P = .86).
Twenty-six subjects were randomly selected for comparison of utilities provided by the chained and traditional gamble; 21 ranked the same outcome, meningitis with deafness, as the fifth outcome. The mean of the chained utilities for these was 0.8477. The mean for the traditional gamble utilities was 0.8003. These values were not significantly different (P = .17).
If we accept that the chances of permanent sequelae from occult bacteremia are very small, as the best evidence in the literature indicates, then the important question is: what risks are parents willing to take to avoid outcomes in the treatment of young febrile children that may be transiently painful and inconvenient for their children? We found extremely high median utility values for our outcomes of occult bacteremia without permanent sequelae. In fact, the median utility values for blood drawn, local infection, and hospitalization were equal to 1, the standard value for perfect health, implying that parents will reject any risk of outcome with severe permanent sequelae. Our methods limited final utility values to 6 decimal places. This indicates that most parents we interviewed would prefer their child have blood drawn (or, for that matter, have a local infection that resolves or be hospitalized for intravenous antibiotics) rather than face a one in a million chance of death. For most of these parents, venipuncture provides essentially free information. Even for those parents whose utility for blood drawing is not 1, it can be asserted that many would prefer blood drawing. If the disutility of blood drawing (which equals 1 minus utility of blood drawing) is less than the probability of death, then blood drawing would still be preferred. Using the conservative estimate from the literature that 2.7% of the 3% of children with occult bacteremia will develop meningitis and about 8% of these will die19 yields a probability of death from meningitis of 0.00006. Parents with a disutility for blood drawing less than 0.00006 will prefer blood to be drawn. In our sample, 84% of parents had disutilities for blood drawing less than this. This is likely an underestimate, as it does not consider the disutility of other permanent sequelae, such as deafness and brain damage.
If venipuncture provides free information, the treatment decisions become very simple from the parents' perspective. We therefore considered threats to the validity of this finding that the median utility is one. One concern is that our method of chained gambles can inflate the utilities of higher-ranked outcomes.20 In fact, in examining a subset of subjects' utilities for deafness, we found that the chained gambles yielded a slightly higher mean utility than the traditional gamble (chained utility, 0.8477; traditional utility, 0.8003; P = .17). Recalculating utilities for higher-ranked outcomes for this subset of patients based on their value for deafness from the traditional gamble, we saw only minimal changes in mean values (for blood drawn, 0.9990 vs 0.9996; for local infection, 0.9984 vs 0.9996; and for hospitalization, 0.9984 vs 0.9992) and no difference in the median values, which remained one. Furthermore, chaining achieved the primary goal of keeping responses to individual gambles within the central 80% of the probability scale. Fifty percent of all the individual gambles yielded probabilities between 0.10 and 0.90 and therefore fell within the central 80% of the probability scale, where values are believed to be most accurate.12 Without chaining, the number falling in that region would likely have been considerably lower, threatening the accuracy of the final utility values.
Another concern is generalizability. Values from our sample of parents might not represent the values of other parents. We found very similar utilities for the parents of young febrile children who were potentially at risk for occult bacteremia compared with parents of afebrile children who were not at risk for occult bacteremia (Table 4). Furthermore, utilities for parents of children with some experience with outcomes did not differ appreciably from those without experience. This suggests that the results are representative of parents presenting to an urban emergency department. However, the extremely high utilities for outcomes that involve invasive procedures or hospitalization may reflect parents' expectations when seen in the emergency department. Parents presenting to their pediatrician's office may have different expectations and different utilities. It must be noted that our sample size is inadequate for detecting small differences between groups. However, the observed differences between groups were far less than 0.1 (Table 4), which has been used as a measure of clinically significant utility difference between groups.12 We do not believe these extremely small differences between the groups are clinically significant. For a single outcome, severe brain damage, a utility difference of 0.1 was found between the febrile and afebrile groups (0.3536 for the febrile group vs 0.4628 for the afebrile group; P = .16). Given the sample size, our power to detect this difference was 0.26,21 and it is likely that with more subjects enrolled this difference would have been significant. In light of the similarities of utilities for all other outcomes (Table 4), we believe that this one difference between the utilities of the febrile and afebrile groups is not clinically significant. Rather, we were impressed that having or not having a febrile child at the time of interview did not seem to matter in the parents' assignment of utilities.
Despite the fact that most parents in our study would consider blood drawing to be of minimal concern (disutility, very low or 0) that it should be performed, approximately 16% of individuals had values that could indicate a different preference. For these individuals, treatment decisions are not so clear-cut; they may prefer a nontest treatment strategy. This variability has been noted in other studies of patients' utilities.16,22 Variability in utilities can have important implications in decisions regarding treatment.23 If parents' utilities for outcomes in occult bacteremia are shown through decision analysis to influence optimal strategies for treaetment, it may be important for physicians to consider parents' preferences in their treatment decisions.
The extreme degree of risk aversion we have demonstrated contrasts with previous descriptions of parents' values for outcomes. Kramer et al9 believed that parents "emphasize the short-term pain, discomfort, and inconvenience of the tests and are willing to risk rare but severe long-term morbidity to avoid short-term risks of testing." However, in characterizing parents as risk-seeking, risk-averse, or risk-neutral, Kramer et al relied on a multiattribute utility model based on values derived from categorical scaling methods. Categorical scaling involves placing an outcome along a VAS as a measure of its value. It has been shown to correlate with utilities from standard gamble methods, but as it does not incorporate any measure of risk preference it is likely to yield significantly lower values than standard gamble.20,24 Therefore, a multiattribute model that relies on this method to calculate utilities could tend to make decision makers (in this case parents) seem more risk-seeking than they actually are. Furthermore, their method of grouping outcomes prevented parents from evaluating outcomes across the entire spectrum of possibilities. Oppenheim et al10 investigated parents' preference for treatment strategies of young febrile children. They found that most parents chose a strategy with increased risk of meningitis. However, it was not clear that parents in that study understood the potential for severe outcomes resulting from meningitis. Parents were told the probability of meningitis but were not given information about the outcomes resulting from meningitis.
There are additional limitations to our study. The method is only applicable to those who can understand and perform the standard gamble. We had a high (20.3%) failure rate for our interview. However, our failure rate is similar to that seen by Gage et al25 (16%) in a Veterans Affairs hospital population that had a much higher percentage of college graduates than did ours (63% vs 22%). Given the multiple gambles evaluated in our interview and our subject population, we believe the failure rate is reasonable. Finally, utility values are only one aspect of analyzing the decision process. High parental utilities do not demonstrate cost-effectiveness from the perspective of any payer, including parents who might be paying out of pocket for care. However, incorporation of the utility values into a decision model with current estimates of probabilities for pneumococcal bacteremia and its sequelae will provide insight into parents' preferences for treatment strategy.
Our assessment of utilities for outcomes of occult bacteremia yielded extremely high mean and median values for outcomes without permanent sequelae. These utilities for outcomes suggest that many parents presenting to an emergency department may rationally prefer painful transient experiences, including venipuncture, for their children rather than risk even very rare chances of severe outcomes.
Editor's Note: The results of this study are not surprising, but they are reassuring about parents.—Catherine D. DeAngelis, MD
Accepted for publication June 7, 1999.
Abstract presented at the Pediatric Academic Societies' Annual Meeting, New Orleans, La, May 4, 1998.
Reprints: Jonathan E. Bennett, MD, Division of Emergency Medicine, St Louis Children's Hospital, One Children's Place, Room 4S50, St Louis, MO 63110.
Bennett JE, Sumner W, Downs SM, Jaffe DM. Parents' Utilities for Outcomes of Occult Bacteremia. Arch Pediatr Adolesc Med. 2000;154(1):43-48. doi:10-1001/pubs.Pediatr Adolesc Med.-ISSN-1072-4710-154-1-poa9043