Bundy DG, Berkoff MC, Ito KE, Rosenthal MS, Weinberger M. Interpreting Subgroup AnalysesIs a School-Based Asthma Treatment Program's Effect Modified by Secondhand Smoke Exposure?. Arch Pediatr Adolesc Med. 2004;158(5):469–471. doi:10.1001/archpedi.158.5.469
DIMITRI A.CHRISTAKISMD, MPHHAROLD P.LEHMANNMD, PhD
Halterman et al1 present a randomized controlled trial (RCT) of a school-based asthma treatment program in this issue of the ARCHIVES. The authors enrolled 184 children with asthma 3 to 7 years of age from 1 school district. Children were identified via a school health form and were eligible for enrollment if at least mild persistent asthma was confirmed and the primary care physician (PCP) agreed with the need for daily inhaled steroids. After obtaining consent from the family and PCP and completing a baseline assessment, eligible children were randomized to the school-based intervention or control group. In both groups, parents and PCPs were notified of the child's asthma severity. On weekdays, school nurses administered study-provided inhaled steroids to the intervention group children; a second steroid inhaler was given to families for home use on nonschool days. Inhaled steroids were recommended but neither provided nor directly administered in the control group. Family report of the number of symptom-free days during the previous 2 weeks was the primary outcome measure. Halterman et al1 found no difference between the intervention and control groups in the primary outcome but did find statistically significant differences in 2 secondary outcomes. Stratification by secondhand smoke exposure resulted in statistically significant differences on multiple measures, all favoring the intervention.
Using standardized criteria for evaluating RCTs,2,3 we examined issues related to validity, treatment effect, generalizability, and subgroup analyses.
After completing baseline assessments, children were randomized to intervention and control groups in blocked groups of 6, stratified by current use of preventive medications, using cards generated from a random-number table. Cards remained in sealed, opaque envelopes until completion of the baseline assessment. Siblings were assigned to the same group.
Randomization resulted in comparable groups with regard to sociodemographic variables, maintenance medication use, presence of a smoker in the home, and baseline asthma severity (Halterman et al1 Table 1). Not included in Table 1 is the distribution of sibling pairs or the presence of additional environmental exposures such as domestic pets, either of which may have affected the results if the distribution were unequal between groups.
Children enrolled for longer than 1 month were analyzed in the groups to which they were randomized. The 4 subjects who participated less than 1 month, all of whom were in the intervention group, were excluded from the analysis owing to lack of data. This low attrition rate likely had minimal effect on the overall study outcomes.
School personnel, participating families and children, and PCPs were aware of their group assignment. Follow-up interviews were conducted by personnel blinded to each child's group allocation, which is necessary for unbiased collection of outcomes data.
To identify the causal agent, it is important to measure the specific components of the program that were delivered to each group. Patients in the intervention group received a single, nurse-administered dose of fluticasone propionate (110 µg/puff) each school day, regardless of their symptoms or severity of asthma. Fluticasone was provided to the intervention group at no cost, for both home and school use. In contrast, asthma medication prescription, titration, and education were managed by the PCP in the control group. Lack of access to a PCP and difficulty obtaining prescribed medications may have accounted for differences between groups. Children in the control group may have received a different dose of the medication if improper technique occurred more frequently with parent-administered doses compared with nurse-administered doses. While receiving the medication at school, the intervention group also may have experienced a different level of support and teaching regarding asthma than those in the control group. The authors appropriately point out, however, that this RCT tested a system change and not simply whether inhaled corticosteroids are more effectively delivered at home or in school. While the groups may have been treated similarly outside the system change, it is not clear whether simply increasing access to medications or having a monitored nurse accounted for differences between groups.
The primary outcome measure of mean symptom-free days did not differ between the 2 groups. Sample-size calculations were based on 80% power to detect a difference of 1 symptom-free day per 2-week period, assuming a standard deviation of 2.5 days. Post hoc calculations indicate that 302 children would have been needed for this RCT to have had 80% power to detect a difference of 1 symptom-free day, given the observed standard deviation (3.1 days). The authors report a statistically significant increase in mean symptom-free days during the early winter months in the intervention group compared with the control group (9.2 vs 7.3; P = .02). Early enthusiasm for study participation, a seasonal change in environmental triggers, or the increased risk of spurious findings when analyzing multiple time points may explain this result. Statistically significant secondary outcomes included a decreased mean number of school absences for asthma (6.8 vs 8.8; P = .047) and greater improvement in mean parental quality of life scores (0.63 vs 0.24; P = .047) in the intervention vs the control group. These results remained significant after controlling for potential confounders.
Population characteristics and once-daily medication dosage limit the generalizability of this study. The sample population in this study consisted of predominantly African American and Latino children aged 3 to 7 years with mild to severe persistent asthma from families of low socioeconomic status in an urban setting. Whether these findings would be similar among children from other demographic groups is unknown. However, adherence with inhaled maintenance medications is poor among children generally,4,5 suggesting that interventions aimed at improving adherence may be widely beneficial.
Although some studies suggest equivalence of once-daily and twice-daily fluticasone treatment, once-daily fluticasone therapy is arguably not the standard of care and requires further research.6 However, decreased efficacy of once-daily therapy would result in smaller observed differences in outcomes between intervention and control groups if more patients in the control group received twice-daily therapy. The potential differential effect of once-daily vs twice-daily therapy on symptom control in children exposed to secondhand smoke is unknown.
The outcomes chosen by Halterman et al1 were clinically important. Parent-reported symptom-free days, use of rescue inhalers, school absenteeism, parental quality of life, acute care visits, and hospitalizations provide a multidimensional measure of asthma outcomes. There were no physiologic measures of asthma severity, such as peak flow or pulmonary-function testing. The authors correctly note, however, that parental-report measures are a component of the National Heart, Lung, and Blood Institute (Bethesda, Md) criteria for defining asthma severity.7
Several other factors should be considered when evaluating the generalizability of this study. While providing daily maintenance medication by school nurses eases the treatment burden on the parent and may increase adherence with medications, it results in an increased, uncompensated workload for the school nurses. Additionally, the nurses knew that they were being monitored as part of the study. It is unknown whether the same effects would be observed if the program were implemented in the absence of the investigation. Shifting the focus of responsibility for maintenance asthma treatment to schools may also decrease parental sense of involvement in asthma care, increasing the likelihood of missed weekend medication doses or inconsistent home monitoring of asthma symptoms. It may also decrease opportunities for titration of medication by decreasing parent and PCP involvement in care.
Despite showing overall differences between the intervention and control groups for 2 secondary measures, Halterman et al1 place strong emphasis on the subgroup differences observed between children exposed and unexposed to secondhand smoke at home, as evidenced by its inclusion in the title. Subgroup analyses are common in clinical trials, but inferences drawn from such analyses must be carefully scrutinized. Next, we examine the subgroup analysis in the present RCT using standardized criteria: (1) the source of subgroup comparisons, (2) the timing and number of hypothesized subgroup effects, (3) effect size and statistical significance, and (4) indirect evidence.8,9
Both subgroups were enrolled within the same study using identical eligibility criteria; thus, one could expect to have similar distributions of unmeasured confounders, which strengthens subsequent inferences. In contrast, when making subgroup comparisons across multiple studies, separating true subgroup differences from the effects of unmeasured confounders is problematic owing to variation in enrollment criteria and study design.
The authors describe their subgroup comparison as a "single post hoc analysis," implying that the hypothesis did not precede the analysis. Planned subgroup analyses are preferred for hypothesis testing because investigators can account for these analyses when determining the sample size. Under those conditions, there is a strong rationale for the prominence of the subgroup analysis. Post hoc analyses, in contrast, should be considered hypothesis generating, leading to future studies for formal evaluation. It must be understood that if there is benefit in some subgroups, there are likely other subgroups in which the intervention produced worse outcomes. In testing only a single subgroup, Halterman et al1 reduced the hazard of reporting spurious subgroup differences resulting from chance, which increases as the number of subgroups tested increases. Adjusting the P value for multiple comparisons is an additional protection against the risk of type I error.
In the Halterman et al1 article, Table 3 summarizes the overall and subgroup effects for the intervention and control arms. The magnitude of subgroup effects varied by asthma-related outcome and was statistically significant for 5 of them. For example, among children unexposed to secondhand smoke, those in the intervention group averaged 1 more symptom-free day in the previous 2 weeks than those in the control group (11.5 vs 10.5; P <.046). Likewise, among children unexposed to secondhand smoke, control group children were 2.5 times more likely than those in the intervention group to have 3 or more acute care visits (32% vs 13%; P <.03). Neither of these findings was statistically significant in the overall analysis. In general, larger differences in subgroup effects are more likely to be true differences.
We found 1 study comparing the effectiveness of inhaled steroids in children with asthma stratified by secondhand smoke exposure. Soussan et al10 followed children using inhaled steroids for 3 years and found that those exposed to secondhand smoke were less likely to achieve symptom control and to reduce peak flow variability. Evidence from the adult literature suggests that inhaled steroids may be ineffective in smokers. In an RCT of inhaled steroids in smokers and nonsmokers with asthma, Chalmers et al11 observed that only nonsmoking patients had improvements in pulmonary-function tests. In a trial of smokers randomized to receive inhaled steroids or a placebo, Cox et al12 found no difference between groups in pulmonary-function tests or airway neutrophilia. Whether tobacco smoke modifies the effect of inhaled steroids in similar ways in children and whether this effect extends to secondhand smoke is unclear.
This well-designed RCT found that a school-based system change for the provision of inhaled corticosteroids to asthmatic children had a minimal effect on outcomes; the effect may be greater among children not exposed to secondhand smoke. Although the subgroup analysis was post hoc, other criteria for sound subgroup analyses were met. The comparison was made within a single study, the subgroup effect was one of a small number of hypothesized effects tested, the magnitude of the effect was moderate and statistically significant, and there was indirect evidence of biological plausibility.
We believe this study provides direction for future research in pediatric asthma. First, it demonstrates that school-based asthma treatment programs are feasible, though their benefits may be modest. Future research may focus on alternative school-based treatment strategies, perhaps incorporating asthma education. Second, the asthma status of both intervention and control group children improved during the study. The control group improvement suggests that provider notification of asthma severity, increased parental awareness of asthma symptoms via symptom diaries, and review during monthly phone calls may improve outcomes. Finally, future research on inhaled steroids for pediatric asthma should be designed prospectively to test whether parental smoking is indeed modifying the beneficial effects. If this preliminary hypothesis is borne out, it would suggest that improved medication adherence will not lead to improved outcomes for asthmatic children exposed to secondhand smoke.
Corresponding author: David G. Bundy, MD, MPH, University of North Carolina, 5034 Old Clinic Bldg, CB7105, Chapel Hill, NC 27599-7105 (e-mail: firstname.lastname@example.org).
Dr Ito holds stock in GlaxoSmithKline, Research Triangle Park, NC, which manufactures fluticasone.