Informing the Uninformed: Optimizing the Consent Message Using a Fractional Factorial Design

Importance  Research information should be presented in a manner that promotes understanding. However, many parents and research subjects have difficulty understanding and making informed decisions.

Objective  To examine the effect of different communication strategies on parental understanding of research information.

Design  Observational study from January 2010 to June 2012 using a fractional factorial design.

Setting  Large tertiary care children’s hospital.

Participants  Six hundred forty parents of children scheduled for elective surgery.

Interventions  Parents were randomized to receive information about a hypothetical pain trial presented in 1 of 16 consent documents containing different combinations of 5 selected communication strategies (ie, length, readability, processability [formatting], graphical display, and supplemental verbal disclosure).

Main Outcome and Measures  Parents were interviewed to determine their understanding of the study elements (eg, protocol and alternatives) and their gist (main point) and verbatim (actual) understanding of the risks and benefits.

Results  Main effects for understanding were found for processability, readability, message length, use of graphics, and verbal discussion. Consent documents with high processability, eighth-grade reading level, and graphics resulted in significantly greater gist and verbatim understanding compared with forms without these attributes (mean difference, 0.57; 95% CI, 0.26-0.88, number of correct responses of 7 and mean difference, 0.54; 95% CI,0.20-0.88, number of correct responses of 4 for gist and verbatim, respectively).

Conclusions and Relevance  Results identified several communication strategy combinations that improved parents’ understanding of research information. Adoption of these active strategies by investigators, clinicians, institutional review boards, and study sponsors represents a simple, practical, and inexpensive means to optimize the consent message and enhance parental, participant, and patient understanding.

As proxy decision makers for their children, adequate parental informed consent is of paramount importance in the research and health care settings.¹ For most studies, the standard vehicle for the disclosure of research information is the consent form, although the process of consent also requires a meaningful dialogue between the investigator and parent or participant.

As part of the consent process, federal regulations mandate that research participants receive information regarding several core elements of consent (eg, risks and protocol) in a manner that promotes understanding.² Despite this, studies show that consent documents are often difficult for the average individual to understand and that verbal disclosure is quite variable.^3-7 This is important because a lack of understanding may result in the unknowing acceptance of potential risk or nonparticipation or may jeopardize the safety of participants who are unable to follow a research protocol.

The observations that understanding of consent information is often inadequate suggest that the current approaches to presenting study information are not always optimal. Indeed, although a variety of interventions to improve participant understanding have been tested, not all have been uniformly effective.⁸ The reasons for this are multifactorial but the fact that research and medical information are often presented in a “one-size-fits-all” format ignores important differences in the way that individuals assimilate and understand complex information. Furthermore, many studies^9-12 describe just a single intervention (eg, shorter forms) such that the potential benefits of multiple interventions on understanding have not been fully explored. Finally, many of these studies have focused solely on the consent document rather than the process as a whole.⁸ The hypothesis to be tested, therefore, was that different combinations of active communication strategies (written and verbal) would improve parents’ understanding of consent information and, thus, optimize the informed consent process.

The study was approved by the University of Michigan institutional review board with a waiver of written informed consent. Parents (>18 years) of children who were scheduled for elective surgery were included.

A series of 16 consent forms were developed that included different combinations of 5 selected communication strategies. Each form was based on the University of Michigan institutional review board consent template and described a hypothetical clinical trial examining the efficacy and safety of transitioning children with moderate to severe pain from a short-acting to a “new” long-acting pain medication (“Painaway”). Prior to distribution, each consent form was reviewed by multiple experts in informed consent and lay individuals for consistency of content across documents.

Based on the extant literature and our previous work, we selected 5 communication strategies that have been shown individually to improve subjects’ understanding of consent information. The 5 selected strategies (each with 2 levels) included:

1. Readability (eighth-grade vs 12th-grade reading level).^13,14

2. Processability/formatting (low vs high).^7,12 Techniques for improving processability included larger font (size 14), more white space, and highlighting using bulleting, bolding, and underlining. Processability was measured using the techniques developed by Irwin and Davis¹⁵ and adapted by Philipson et al¹⁶ for use with consent forms.

3. Graphic display of risks and benefits (graphs vs text). Pictographs were chosen to display risks and benefits based on previous research showing their effectiveness.^17-20

4. Verbal disclosure (yes vs no).^7,21-23 Subjects received either a supplemental verbal description of the study using a standard script or no verbal description.

5. Message length (short vs long).^7,10,24 Simplified vs more detail. On average, the longer forms were 3 to 4 pages longer than the shorter forms.

Regardless of which strategy combinations were used, each form contained all the elements of consent such that content between documents was not compromised.

Because a full factorial design for 5 factors would require an unwieldy 2⁵ = 32 experimental groups, we used instead a 16-group fractional factorial design in which parents were randomized to receive information about the pain study in 1 of 16 different consent forms, each of which varied in terms of the message-level combinations (Table 1). To determine the operational effect of each of the 5 strategies, half of the parents were randomized to one level of the factor and half to the other level and the outcomes (ie, understanding) were compared. Fractional factorial designs have been used effectively in engineering applications^25-27; however, their usefulness in behavioral studies has only recently been recognized.²⁸ The advantage of fractional factorial designs is that they allow for the study of a large number of factors using fewer cells than required by a full factorial design (ie, 16 vs 32). Thus, we chose to use a resolution V fractional factorial design with 16 groups that would allow us to estimate all main effects and all prespecified 2-factor interactions among the 5 communication strategies.

Parents were approached in the preoperative waiting area by trained research assistants and told that we were evaluating the quality of our consent process. Parents were also told that their child would not be participating in an actual study but that they should consider the information as if real. Although the study represented a hypothetical study, it was nevertheless presented to simulate our standard practice for consenting parents (ie, place, time to read the consent, and environment). Parents received a $5 gift card for their participation.

Parents were interviewed to measure their understanding of 11 required elements of consent.² The semistructured interview was based on the Deaconess Informed Consent Comprehension Test²⁹ and followed the format described previously.^3,7,30 Outcome measures included understanding of the study purpose, procedures, alternatives, direct and indirect benefits, risks, voluntariness, freedom to withdraw, study duration, contact information, and confidentiality. The open-ended responses were written down verbatim by trained research assistants. The interviewer was allowed to clarify individual questions and prompt parents for additional information but could not offer specific details. The transcribed responses were scored independently by 2 assessors who knew the study but were blinded as to which consent form was administered. The scoring system used a scale of 0 to 2 based on the parent having no (0), partial (1), or complete (2) understanding.⁷

Gist understanding refers to the ability to understand the essential meaning of the risks and benefits.^31,32 Seven gist questions were included, 1 of which was: “If a child received Painaway, which of the following is most likely: (1) good pain relief, (2) nausea and vomiting, (3) constipation, (4) slowed breathing.” Gist understanding was defined as the ability to respond correctly to more than 5 questions of 7. This threshold definition was based on previous studies^17,19; however, as validation, a sensitivity analysis was conducted and found qualitatively similar results using either a continuous measure of understanding or different thresholds, eg, 7 of 7 correct answers.

Verbatim understanding refers to the ability to correctly report the actual risk/benefit statistics.^31,32 Four verbatim questions were included and required parents to respond to the question, “If 100 children took Painaway, how many would experience: (1) good pain relief, (2) nausea and vomiting, (3) constipation, and (4) excessive sleepiness?” Per previous work, verbatim understanding was defined as the ability to correctly respond to 3 or more of 4 verbatim questions.^17,19

Parents completed a questionnaire and 2 validated instruments regarding factors that influenced their understanding of the consent document.

1. Perceptions of the risks, benefits, and study importance (0-10 scale, where 10 = high).

2. Satisfaction with the consent document format (5-point Likert scale of “extremely dissatisfied” to “extremely satisfied”).

3. Consent form factors:

   • Degree to which parents listened (if applicable) to the verbal disclosure and read the consent document (ie, completely, partially, or not at all).

   • Clarity of the information (ie, not at all clear, fairly clear, or very clear).

   • Amount of information (ie, too little, just right, or too much).

   • Readability (ie, very difficult to read, about right, or very easy to read).

1. Sociodemographics: age, sex, race/ethnicity, education, and parent/child prior research participation.

2. Medical literacy: measured using the validated Rapid Estimate of Adult Literacy in Medicine instrument.³³

3. Numeracy: measured using the validated 8-item Subjective Numeracy Scale.^34,35

4. Parental anxiety: measured using a scale of 0 to 10 (10 = extremely anxious).^36,37

On completion of the study, parents were shown 2 consent documents representing 2 extremes of the written strategies (ie, eighth-grade reading level, high processability, short, and graphics vs 12th-grade reading level, low processability, long, and no graphics) and asked which they preferred (eAppendix 1 and eAppendix 2 in Supplement).

Data were analyzed using PASW (SPSS) and R software (http://www.r-project.org/). Prior to the principal analyses, 1-way analysis of variance was used to determine if randomization was successful in creating equivalent groups. Because the threshold definitions of understanding, eg, 3 or more correct answers of 4, are considered discrete data, they did not satisfy the usual assumptions for analysis of variance and were thus analyzed using logistic regression techniques. Although these definitions have been used previously,^17,19 we performed sensitivity analyses to ensure that different threshold definitions produced qualitatively similar results. For example, the number of correct responses for verbatim understanding ranged from 0 to 4. Comparisons were thus made between 0 and 3 vs 4 correct answers; 0 to 2 vs 3 to 4 correct answers; and 0 to 1 vs 2 to 4 correct answers. Similar analyses were conducted for the other outcomes and confirmed that different thresholds produced qualitatively similar results.

The final models were determined using the Akaike information criterion model-selection criterion.³⁸ The Akaike information criterion selects the best-fitting model from other candidate models in minimizing information loss. Interrater reliability for the assessors’ scores of understanding was measured using the κ statistic. κ Scores for each of the elements of consent ranged from 0.76 to 1.0 (P < .001) revealing very good/excellent agreement. Data are described as mean differences and odds ratios with 95% confidence intervals.

Sample size was based on previous data comparing parental understanding of consent information using different individual communication strategies, eg, use of different graphics and high vs low processability.^7,20 Based on these data, we considered it important to detect an effect size of 0.5 between levels of each factor. This estimate required at least 640 subjects, ie, 40 subjects per cell in the 16-cell fractional factorial design (α = .05 and β = 0.1, 2-sided).

A total of 871 parents were approached to participate in this study, of whom 209 declined and 22 withdrew. Data are presented for 640 parents. There were no differences in demographics between the 16 consent groups. Table 2 provides a summary of the entire sample.

Overall, 182 parents (28.5%) reported that they had read “most” and 441 (69.0%) “all” of the consent form to which they were randomized. Short forms were more likely to be read completely compared with long forms (odds ratio, 1.75; 95% CI, 1.23-2.49). Of those who did not read the entire form, 33 (5.2%) skipped the study purpose; 31 (4.9%), the procedures; 30 (4.7%), the benefits; 84 (13.2%), privacy issues; and 108 (17.0%), contact information.

Table 3 and Table 4 describe the logistic regression analyses for verbatim and gist understanding, respectively. As shown in Table 3, high processability and inclusion of graphs had significant positive effects on verbatim understanding as indicated by their positive coefficients. The odds ratio for inclusion of graphs, for example, showed that the odds of verbatim understanding were increased by 50% if graphs were used. Longer consent forms, on the other hand, had a negative effect on verbatim understanding. For gist understanding (Table 4), high processability, inclusion of graphs, and verbal disclosure had significant positive effects, whereas readability (12th-grade level) had a negative effect. Logistic regression analyses for understanding of the individual elements of consent showed that high processability had significant positive effects on understanding of the study’s purpose, procedures, risks, benefits, and study duration. Use of graphs had positive effects on the understanding of risks and alternatives and supplemental verbal disclosure had a positive effect on understanding of the procedures. In all regression models, high numeracy and white race predicted improved understanding of the consent information.

Based on the results of the regression models, we assigned positive attributes to eighth-grade reading level, high processability, graphics, verbal disclosure, and shortened forms. We then examined the effect of the number of positive attributes (1, 3, or 5) per consent form on parents’ understanding and perceptions (Table 5). Results showed that although inclusion of all 5 positive attributes resulted in a trend toward greatest understanding, forms with at least 3 positive message attributes were sufficient to improve understanding significantly over those containing only 1 positive attribute. The number of positive attributes had no effect on understanding of voluntariness, freedom to withdraw, confidentiality, study duration, and contact information. Given that 3 positive attributes were sufficient to improve understanding, the effect of combining the 3 messages with the strongest main effects, ie, eighth-grade reading level, high processability, and graphics, was examined. Forms containing these specific attributes resulted in significantly greater gist and verbatim understanding compared with forms without these attributes (mean difference, 0.57; 95% CI, 0.26-0.88, number of correct responses of 7 and mean difference, 0.54; 95% CI, 0.20-0.88, number of correct responses of 4 for gist and verbatim, respectively). Similarly, understanding of the study purpose (mean difference, 0.22; 95% CI, 0.07-0.36, number of correct responses of 2), the risks (mean difference, 0.22; 95% CI, 0.07-0.38, number of correct responses of 2), and benefits (mean difference, 0.17; 95% CI, 0.005-0.33, number of correct responses of 2) were better understood when these 3 attributes were included.

When shown the 2 documents with contrasting attributes (form 8 vs form 10) (Table 1 and eAppendix 1 and eAppendix 2 in Supplement), 460 parents (74.3%) preferred the consent form with the positive attributes (form 8). Comments included “less words, easier to understand, simpler, like the bullet points, key points more obvious.” Those who preferred form 10 reported that they preferred more information and detail.

Previously, Flory and Emanuel⁸ conducted a systematic review of interventions designed to improve understanding of informed consent for research. This review highlighted several individual interventions used in the present study that improved understanding of consent including enhanced consent forms (ie, improved readability, reduced length, and formatting) and verbal disclosure.^9,10,12,23 However, this review also noted that these interventions were not consistently effective, suggesting that any single intervention cannot be “all things to all people.” To date, there have been limited data with respect to the effectiveness of combining interventions; thus, our study is important in identifying specific strategies that when combined enhance understanding on a more consistent level.

The present study identified several active communication strategies that improved parental understanding of consent information. Furthermore, data showed that consent forms containing at least 3 active communication strategies resulted in enhanced understanding. In particular, consent forms written at the eighth-grade level with high processability and use of pictographs provided the best format for understanding risk/benefit statistics. Document processability was a strong and consistent predictor of understanding in our study. Improved processability was achieved simply by using larger font, more white space, and bolding and underlining important information. Previously, we showed that consent/assent forms with high processability significantly improved understanding of research information and were preferred by 82% of parents and children compared with a standard form.^7,39 The use of risk/benefit graphics also resulted in improved understanding of risks. We chose pictographs as the graphic of choice because previous work suggests that this format is both preferred and better understood compared with text and other graphical formats.^17,19,20 Graphical depiction of risks and benefits may work because they have greater visual salience, require less cognitive effort, and are easier to understand.^40-43

It was surprising that health literacy was not an independent predictor of understanding given that previous studies have shown the importance of health literacy in understanding health information. A likely reason for this was that our population was skewed toward those with good health literacy and this likely confounded the results. Despite this, the consent forms written at an eighth-grade level were better understood compared with those written at the 12th-grade level, and this reemphasizes the importance of reducing the grade reading level of these forms.

Numeracy, however, was a significant predictor of understanding. Given that innumerate individuals are more likely to make risk/benefit trade-offs based on emotion or trust rather than numbers,⁴⁴ these results are significant in emphasizing the importance of improving the presentation of risk/benefit statistics to those with low numeracy. Overall, white parents had higher numeracy and literacy abilities compared with minority parents and were more likely to have been educated beyond high school. However, even when controlling for these factors, white parents had better understanding compared with minority parents. These results are important in that a lack of understanding may contribute to greater distrust or disparities in research participation.^45,46

There are limitations to our approach that must be recognized. First, this study was based on a hypothetical study and as such may not fully replicate the real-life situation. To obviate this concern, the consent forms and the process of consent for this study mimicked our routine research practice. Furthermore, there is precedence for the use of simulated methods and strong evidence showing that behaviors based on real and simulated methods are highly correlated.^47,48 Second, while the supplemental verbal disclosure used in this study was standardized, we recognize that, in practice, more or less information may be given depending on the investigator-subject interaction.

This study highlights the importance of using several different active communication strategies in the design of clinical consent forms. Results suggest that incorporation of at least 3 of the identified active strategies serves to significantly improve understanding of the information and perceptions of the message delivery. Although these results are based on parents’ understanding of pediatric consent documents, it is reasonable to assume that they would also be relevant to consent forms for adult research subjects and for patients who consent to treatment. We therefore recommend that investigators, clinicians, institutional review boards, and study sponsors consider inclusion of these active strategies as a simple, effective, and inexpensive means to optimize the consent message and enhance subject understanding and participation.

Corresponding Author: Alan R. Tait, PhD, Department of Anesthesiology, University of Michigan Health System, 1500 E Medical Center Dr, Ann Arbor, MI 48109 (atait@umich.edu).

Accepted for Publication: November 10, 2012.

Published Online: May 13, 2013. doi: 10.1001/jamapediatrics.2013.1385

Author Contributions:Study concept and design: Tait, Voepel-Lewis, and Fagerlin.

Acquisition of data: Voepel-Lewis.

Analysis and interpretation of data: Tait, Voepel-Lewis, Nair, Narisetty, and Fagerlin.

Drafting of the manuscript: Tait, Nair, and Narisetty.

Critical revision of the manuscript for important intellectual content: Tait, Voepel-Lewis, Nair, Narisetty, and Fagerlin.

Statistical analysis: Tait, Nair, and Narisetty.

Study supervision: Tait and Voepel-Lewis.

Conflict of Interest Disclosures: None reported.

Funding/Support: This work was supported by National Institutes of Health Eunice Kennedy Shriver National Institute of Child Health grant R01 HD053594 (Dr Tait).

Disclaimer: The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs.