Can an Information Prescription Change Parental Attitudes and Behaviors Related to Using the Internet for Health Information Resources?

The randomized controlled study by D'Alessandro et al¹ in this issue of the ARCHIVES examines a novel educational intervention that entails providing an information prescription (IP) to parents at their children's well-child visits. Providing the IP and the accompanying guidance about its use can be considered a type of therapy. We appraise the study with regard to its validity, its results, and how it will help in caring for patients, using the Users' Guides to the Medical Literature for therapy and prevention published in JAMA.^2,3

Obtaining a sample population is important in determining not only to whom the study may be generalized but also the validity of the results. For example, if all patients enrolled in a study are more likely than average to respond to an intervention, the study may yield a falsely positive result.

Appropriately, in this study, everyone who entered the general pediatrics clinic within a certain period was approached. This yielded 250 families. Of these, 238 were eligible based on the exclusion criteria, and 30 additional families then refused to participate. Ideally, all of the families asked would participate; however, the consent rate of 83.2% is very good for a trial of this nature.⁴

This study was a randomized controlled trial. Proper randomization of groups is perhaps the most essential characteristic of a valid study, helping to ensure that the 2 groups are similar at baseline. The method of randomization in this study was to shuffle the envelopes containing control and intervention group forms and then to pick them out of a box. However, even seemingly random methods can be subject to subtle biases. Recommendations for rigorous group assignment have been described, and additional details from the authors would have been useful.⁵ For example, because the envelopes contained control and intervention group forms, if the respective forms were of different weight or size, the allocation may not have been completely concealed. This may cause the 2 groups to be slightly different at baseline and may or may not change the outcome of a study.

To determine whether 2 groups differ significantly in the studied variable, researchers should ensure that the groups are as similar as possible in all other respects, reducing bias. Randomization does not guarantee that the 2 groups will be similar. In this particular study, the demographics of each group were similar; however, the groups differed significantly in their answers to 3 of the 40 questions posed to them on a prestudy questionnaire. The authors explain that in any randomized trial, it would be expected for approximately 5% of the answers to differ between the randomly assigned groups.

Although it is true that the groups may be randomly different, this is not necessarily acceptable. One must look closely at the characteristics by which the 2 groups differ. If the 2 groups in the study differed in their answers to the questions "Do you get information on health from the Internet?" and "Do you have access to the Internet?" this would significantly impact the validity of the study, even if these were the only answers that differed. Fortunately, in this study, the 3 questions for which the answers differed do not appear to have much impact on the validity.

Blinding both the investigators and the subjects to the treatments increases the validity of any study by eliminating the possibility that the opinions of the investigators or subjects bias the results. In this study, neither investigators nor subjects were blinded to the treatment prescribed. It would be difficult to blind both parties in this study because the IP relies on personal interaction between doctor and patient. Nevertheless, the lack of blinding opens the question of bias: Did the subjects, knowing they were in the intervention group, visit Web sites (or claim to have visited a Web site) in the interest of helping the study, even if they would not have done so normally? Did the investigators treat the intervention and control groups differently in some way that would bias the results?

A low dropout rate is a key factor in assessing the validity of a study. Patients who drop out from each group may represent different outcomes from those who remain in the trial. The researchers did have a low dropout rate. Only 5 patients dropped out of the control group and 6 from the intervention group. However, the researchers do not perform an intention-to-treat analysis.

In essence, an intention-to-treat analysis assumes the worst-case scenario of the data, if all of the dropouts were to have continued with the study. That is, this analysis supposes that dropouts from the control group had favorable outcomes and intervention-group dropouts had unfavorable outcomes.⁶ In this study, using the data presented, we were unable to replicate the authors' P value for their main outcome measure: Did the intervention group use the Internet for general health information more than the control group? Hence, it was not possible to re-analyze the results using an intention-to-treat analysis, which is a more conservative estimate of the effect of the intervention.

The outcomes included changes in parental attitudes and behavior (ie, Internet use). The authors report no difference in parental attitudes regarding the helpfulness of the IP intervention (P = .31). However, the authors suggest a statistically significant increase for the IP group in the use of the Internet for general health information (44.3% vs 33.0%; P = .05) and child health information (39.2% vs 14.0%; P<.001) compared with the control group.

The P value can be thought of as the probability that this observed result (ie, an increase in Internet use by the IP group) is due to chance alone. A more useful measure for clinicians is the 95% confidence interval, which describes the precision of the treatment effect estimate. We were unable to calculate the treatment effect or the 95% confidence interval of the treatment effect owing to ambiguity about the unit of analysis.

The unit of analysis could represent the number of different Web sites or search engines used, the number of times the Internet was used, or the number of families in each group who used the Internet. For example, in the intervention (IP) group, 20 unique Web sites and search engines were used for a total of 77 total uses. The authors note that 44.3% of the intervention group used the Internet, which implies that 43 (44.3%) of the 97 IP families were responsible for these 77 total uses.

In contrast, in the control group, 23 unique Web sites and search engines were used for a total of 28 times. According to the results, because 33% of the control group used the Internet, this implies that 33 (33%) of the 100 control families were responsible for the 28 uses. It is not clear how 33 control families were responsible for only 28 Internet uses. Unless the families were using a shared computer and accessing the Internet at the same time, one would expect the number of Internet uses to be equal to or greater than the number of families reporting use. A clearer explanation of the unit of analysis might help clarify these questions.

When the Internet was accessed, it is striking that the recommended Web sites were used during 51 (66.2%) of 77 of the Internet uses for the IP group but only for 1 (3.6%) of the 28 total uses for the control group. Although the impact of the IP on Internet use is questionable, the impact of the IP on increasing the use of recommended Web sites is much more convincing, despite issues with intention-to-treat analysis and the unit of analysis.

The demographics of the subject population are reported, which is useful in determining the applicability of the results. Patients were recruited from the University of Iowa's faculty general pediatric clinics. Enrollment criteria in this study included parents or guardians willing to enroll, residing with the child, English speaking, having a telephone, and available 2 to 3 weeks later. The study population spoke English only and was largely homogeneous with respect to sex, race, educational level, insurance status, and income. Given this homogeneity, it is unclear to what extent the results are generalizable to other patient populations. Furthermore, as the authors acknowledge, the patient population was higher than the national average on socioeconomic and educational variables.

On other preintervention survey variables, this study population reported that health care providers and the Internet were the most common and preferred sources of health information. There were no subjects without computer access. Most felt that having this IP therapy would be helpful. The high use of the Internet by this study population may affect estimates of applicability. It is unclear whether this therapy would help a parent who did not already use the Internet for health information because most parents in this study were already Internet users as well as Internet users for health information.

A subgroup analysis of those who actually reported using the IP therapy was conducted. The authors acknowledge that there were no preintervention study variables that could predict who would go on to use the IP therapy. This too makes it difficult to estimate to whom the results will likely apply.

The study's primary outcomes were whether the intervention group reported using the Internet for general and child health information more than the control group, whether they reported using the recommended Web sites, and whether they perceived the IP therapy to be valuable enough to request it again in the future and to recommend it to others. No adverse outcomes were specifically quantified or reported.

Ideally, a study of therapy would measure improved health outcomes. Yet, owing to constraints on time and resources, substitute end points are appropriate to use and were used in this study. The authors state and cite in their introduction that well-informed patients have better outcomes than those who are not well informed. To the extent that this is true, a substitute end point of knowledge would be most appropriate to measure. Yet, the primary outcome measures in this study are attitudes and behaviors, not knowledge.

Treatments are indicated when they provide important benefits. Attitudes about the Internet were already favorable in this study population. We cannot know from the data if the outcomes of improved attitudes and behaviors would be important in any other patients who do not already regard the Internet favorably.

We may assume (although this was not specifically reported) that the therapy was generally well tolerated. However, the authors do not discuss potential or actual risks of the treatment studied, nor do they report on potential harms of withholding treatment. We can calculate the number needed to treat in order to prevent nonuse of the Internet for health information. Based on the reported data in this study, 9 people would need to receive the intervention to prevent one event of nonuse of the Internet for general health information.

The treatment is benign with the exception of the time investment. In this study, the intervention took 7 minutes, which should be considered a long time when the average well-child care visit is estimated to be less than 16 minutes.⁷ The time put toward this intervention may be taking away time or resources from another aspect of patient care (eg, developmental assessment or screening tests).

Restricting therapies only to ideal patients may result in harm to those excluded.⁸ Given that there were no preintervention study variables that could predict who would go on to use the IP therapy, it is hard to know how to limit the therapy to only those ideal patients. Future study among a more socioeconomically diverse population may be able to clarify these questions.

This randomized controlled trial of the effectiveness of an IP concerns an innovative educational intervention aimed at changing patient attitudes and behavior. This is a well-conducted study; however, there are limitations to the analysis and generalizability.

The impact of the IP on Internet use is borderline at best because of issues involving how the unit of analysis was defined and the lack of an intention-to-treat analysis. However, the impact of the IP on increasing the use of recommended Web sites is much more convincing. This suggests that the use of an IP alone cannot help parents overcome the barriers to using the Internet for health care–related information; but for those who already use the Internet, the IP can help direct such families to recommended sites.

The IP intervention was studied at an academic practice serving English-speaking families with an above-average socioeconomic and educational level. All patients had access to the Internet. In addition, practices would need to have the resources and time to be able to invest 7 minutes per visit for this intervention. As a result, these findings may not be applicable to many patient populations or general pediatric practices. However, for those practices with significant resources that serve a population already using the Internet, the use of an IP may help families better navigate the Internet to find appropriate health-information resources.

Correspondence: Dr Kind, Children's National Medical Center, Children's Health Center at Good Hope Road, 2501 Good Hope Rd SE, Washington, DC 20020 (tkind@cnmc.org).