Can an Herbal Preparation of Echinacea, Propolis, and Vitamin C Reduce Respiratory Illnesses in Children?

The double-blind placebo, randomized controlled study by Cohen et al¹ investigated the effectiveness of Chizukit (Hadas Corp Ltd, Yokneam, Israel), an herbal preparation of echinacea, propolis, and vitamin C, in preventing respiratory tract infection in children. This study enrolled 430 children, aged 1 to 5 years, from 10 primary care community clinics in Israel. Children were randomized to treatment or placebo groups and followed up for a 12-week period during the respiratory season. There were several primary outcomes, but the main outcomes studied were the number and duration of illness episodes. Children treated with Chizukit had a statistically significant reduction in the number and duration of illness episodes when compared with children treated with a placebo. We appraised the article for validity and generalizability using the Users' Guide to the Medical Literature articles about therapy or prevention.^2,3

The purpose of randomization is to ensure balance between study groups for baseline characteristics. One way of ensuring truly random allocation to treatment groups is to use computer-generated randomization. Bias can still be introduced after randomization if allocation to treatment groups is predictable by study personnel. One way of concealing the randomization procedure is to remove it from the site where patients are being enrolled and evaluated. Optimal randomization procedures were followed in this study, because allocation was done by computer-generated randomization and assignments were stored in sealed envelopes in the pharmaceutical company's pharmacy.

In general, in a randomized clinical trial, it is necessary to determine if any baseline differences between groups may have affected results. We were not able to assess the success of randomization because, other than mean age, the authors did not provide baseline data for the treatment groups. Important demographic variables to consider include age distribution, sex, income, and maternal education level. Baseline information on other variables known to affect rates of acute respiratory illness (day-care attendance, passive exposure to cigarette smoke, history of atopy, and breastfeeding) is also critical. This information is important because differences between groups could have confounded results. Confounding can occur when an extrinsic factor is associated with both the predictor and outcome variables.⁴ It is then difficult to determine if the extrinsic factor itself caused the observed effect. For example, confounding could occur if a relatively larger proportion of children in the placebo group attended day care. The placebo group might have had more illness episodes because of day-care attendance rather than it being a true reflection of the efficacy of Chizukit. Although in this study baseline variables should have been roughly balanced between the 2 groups because of careful randomization, lack of information comparing treatment groups at entry renders the results difficult to evaluate.

Blinding minimizes bias that may be introduced if patients or researchers are aware of the treatment being used by a particular subject. The study was a double-blind trial. Great effort was placed on blinding both subjects and researchers to treatment assignment. The investigators reported that Chizukit and the placebo were identical in appearance, color, flavor, dosage, and packaging, and both were supplied directly by the manufacturer.

Of the 430 children in the study, 328 (76.3%) completed the 12-week follow-up period. It is important to account for all participants. In the Cohen et al¹ article, Figure 1 guides the reader through participant flow in the trial. The numbers in Figure 1, however, are not entirely consistent (215 randomized to Chizukit − 58 dropouts = 157, but the authors reported n = 160; 215 randomized to placebo − 44 dropouts = 171, but the authors reported n = 168). The reason for more than 50% of the children who dropped out in both groups was "noncompliance without any explanation." It would have been helpful if this category had been better defined by the investigators. All dropouts occurred in the first week of treatment.

As a result of high attrition, the authors decided to change from an intention-to-treat analysis to a per-protocol, or efficacy, analysis. An efficacy trial evaluates only participants who complied with the intervention, and an intention-to-treat analysis, or effectiveness trial, analyzes data on all participants—including dropouts—based on their original, random assignment.⁴ An intention-to-treat analysis is more difficult to execute but yields more generalizable results. In this study, dropouts were excluded from the analysis. In addition, adherence to treatment was monitored but not adequately described (for example, were pill counts obtained?). The decision on whether the study is an efficacy vs an effectiveness trial should be made a priori, not as a result of high attrition. This further decreases generalizability of study results.

Detection bias occurs when researchers look more carefully for outcomes in 1 of the 2 groups being compared. In this study, both groups were monitored in the same way, with weekly telephone calls, symptom diaries, and scheduled visits at 4, 8, and 12 weeks.

Significant reductions in the number and duration of upper respiratory tract illness episodes were found in the Chizukit group. The number of children with illness episodes in the placebo and Chizukit groups were 150 (89%) of 168 and 85 (53%) of 160, respectively, which is equivalent to an absolute risk reduction of 36%. Accordingly, approximately 3 children would need to be treated to prevent 1 episode of upper respiratory tract illness. There were 4 other closely related primary outcomes (total number of illness days, number of children with at least 1 episode, number of episodes per child, and days of illness per child) that, as expected, also showed a significant response to treatment.

Many secondary outcomes (acute otitis media, pneumonia, tonsillopharyngitis) were examined, and most were reported significantly less frequently in the Chizukit group. However, the training and qualification of examiners and specific criteria for these diagnoses are not clearly stated. Many children with upper respiratory tract infections have fluid in the middle ear (otitis media with effusion), a condition that is frequently misdiagnosed as acute otitis media.⁵ Assuming that the criteria for diagnosis of acute otitis media were not standardized and that the numerous practitioners involved were not properly trained, the differences between treatment groups may reflect differences in incidence of upper respiratory tract infection. Accordingly, conclusions regarding efficacy of Chizukit for reducing these less rigorously examined outcomes should be made with caution.

The authors report 6 highly related primary outcomes and at least 12 secondary outcomes. In a randomized controlled trial, it is unusual to have so many primary outcomes given the difficulty of studying multiple outcomes accurately. Although the outcomes chosen are clinically relevant to respiratory tract infections in children, specific criteria used for these diagnoses are not clearly stated in the article. For example, parents were asked to contact study personnel to "confirm" the diagnosis of an upper respiratory tract infection, but details on how this confirmation occurred (clinical evaluation vs telephone assessment) are not clearly stated.

The authors did provide their assumptions for the sample size, which included: a probability of upper respiratory tract infection of 18% (based on a previous study of children and adults), a power of 90%, an α of .05, and a hypothesized risk reduction of 50%. Although the accuracy of these estimates may be questioned, the adequacy of the sample size calculation is less relevant because of the significant effect of treatment.

Patients were recruited from primary care clinics in Israel. Demographic characteristics of the children entered, however, are not further described, limiting the applicability of study results.

The incidence of upper respiratory tract infections in the placebo group seems to be relatively high; 94% of children had an illness episode during the 12-week study period, a proportion substantially higher than the anticipated 18%. The high incidence of respiratory tract infections in this group may be related to patient selection (more likely to select frequent visitors to the clinics), identification of illness episodes (increased detection of symptoms in the setting of a research study), or attrition (well patients may have been more likely to drop out). The incidence of upper respiratory tract infection in the study affects efficacy estimates. For example, if the incidence of upper respiratory tract infections would have been 20% and 10% in the placebo and Chizukit groups, respectively, then 10 children would have to be treated with Chizukit to prevent 1 child from having upper respiratory tract infections.

Although Chizukit is not available in the United States, the different components that make up the supplement are. However, the concentrations of active ingredients in many naturopathic remedies have not been standardized.

Chizukit was generally well tolerated and the frequency of adverse effects was not significantly different between treatment groups. Nine children treated with Chizukit experienced mild gastrointestinal symptoms or problems with palatability. Larger studies need to be conducted to fully assess its safety in children.

The results presented in this article suggest that an herbal preparation of echinacea, propolis, and vitamin C is associated with a significant reduction of upper respiratory tract infections in children 1 to 5 years of age. This decrease in upper respiratory tract infections in young children would have significant economic and social ramifications. Shortcomings of the study include lack of data comparing demographic characteristics between treatment groups, large attrition resulting in change from an intention-to-treat to a per-protocol analysis, and lack of clearly defined outcomes. Nevertheless, we find the magnitude of the results compelling, warranting further research.

Corresponding author and reprints: Shilpa Sangvai, MD, General Academic Pediatrics, Children's Hospital of Pittsburgh, 3705 Fifth Ave, DeSoto G-323, Pittsburgh, PA 15213 (e-mail: shilpa.sangvai@chp.edu).