Assessment of Length and Readability of Informed Consent Documents for COVID-19 Vaccine Trials

Key Points

Question  What are the accessibility and understandability for average audiences of current informed consent documents for the COVID-19 vaccine trials?

Findings  This quality improvement study of 4 informed consent documents from 4 major COVID-19 vaccine trials found all to be overly long and complex, exceeding a grade 9 language complexity and requiring a mean of 35 minutes to read the entire document. It was possible to reduce the length of the documents by more than 50% and use more broadly understandable language with a grade 7 or 8 reading level.

Meaning  These findings suggest that informed consent documents may fail to succinctly explain the studies to participants of all reading levels and that it is possible to improve these documents to increase participant accessibility.

Importance  Informed consent is a fundamental element of research ethics. The COVID-19 vaccine trials are high profile trials that have enrolled more than 100 000 participants. Consent documents must be succinct and understandable to ensure informed voluntary participation.

Objective  To assess how well informed consent documents of the COVID-19 vaccine trials achieve the ideal of being succinct and understandable, and to create a shorter, more readable document.

Design, Setting, and Participants  This quality improvement study collected and analyzed the informed consent documents used in 4 COVID-19 vaccine phase III randomized clinical trials to quantitatively assess readability and length and, based on this analysis, created a measurably more accessible informed consent document. Analysis was conducted from October 2020 to January 2021.

Main Outcomes and Measures  The main outcomes were number of words (measured as word count), time-to-read (measured at reading speeds of 175-300 words per minute), language complexity (measured using Flesch-Kincaid Grade Level assessment), and readability (measured using Flesch Reading Ease Score). Secondary outcomes included clarity of how the placebo group could access the vaccine if it is proven safe and effective. The study also examined the length and readability of an improved consent document.

Results  The 4 informed consent documents were a mean (range) of 8333 (7821 to 9340) words long, with a mean (range) 35 (32.6 to 38.9) minutes to read at 240 words per minute. All documents exceeded grade 9 language complexity and scored lower than 60 in the formal reading ease metric, which constitutes difficult. Only 1 document specified that participants in the placebo group might receive vaccine. It was possible to write a document in fewer than 3000 words with a grade 7 to 8 reading level and a formal readability score that was not difficult.

Conclusions and Relevance  These findings suggest that existing COVID-19 vaccine informed consent documents were too long, difficult to read, and exceeded grade 9 in language complexity. It was possible to create a shorter, more readable informed consent document for these trials.

Informed consent is a fundamental protection for research participants. Informed consent documents are one, albeit critical, element in a process that is meant to ensure participant understanding and voluntary participation. Federal regulations on informed consent emphasize that documents should be brief, readable, and prioritize participants’ understanding.¹ Yet, over time, these documents have become longer and more complex.^1,2

The COVID-19 vaccine phase III randomized clinical trials have been the most visible clinical trials in more than 30 years and collectively enrolled more than 100 000 Americans. These trials are occurring after the revision of the federal regulations that emphasize shorter, readable documents.³ The goal of this quality improvement study was to examine how well the informed consent documents achieve the ideal of being succinct and understandable.

This quality improvement study was exempt from institutional review board review because it involved no human research participants. In this quality improvement study, we systematically evaluated the informed consent documents from the AstraZeneca, Johnson & Johnson, Moderna, and Pfizer COVID-19 phase III vaccine randomized clinical trials based on 4 criteria. First, for length, we counted words and calculated the approximate time-to-read based on a typical 240 words per minute (wpm) reading speed (range, 175-300 wpm).⁴ Second, for language complexity, we used a Flesch-Kincaid Grade Level assessment.⁵ Third, for readability, we evaluated the Flesch Reading Ease Score, a 1 to 100 ranking, with scores less than 60 being considered by the Department of Health and Human Services (HHS) as difficult.⁶ Fourth, we assessed how the documents addressed access for placebo groups to the vaccine if it is proven safe and effective.

We wrote a shorter, more readable consent document. This document covers the same substantive material as the original informed consent forms.

All reading analysis was conducted using Readable.⁷ Analysis was conducted from October 2020 to January 2021.

Among the 4 informed consent documents examined, the mean (range) page count was 21.8 (17-25) pages, and the mean (range) word count was 8333 (7821-9340) words (Table 1). At 240 wpm, a participant would need a mean (range) of 34.7 (32.6 to 35.9) minutes to read an informed consent document, not accounting for rereading. Adults with slower reading ability (175 wpm) would require a mean (range) of 47.6 (44.7-53.4) minutes, if they were able to read without stopping.

The language complexity in all the documents exceeded a grade 9 reading level, which is higher than the recommended grade 6 reading level.⁶ Additionally, all the documents had scores of less than 60 in the reading ease metric, with a mean (range) score of 52.4 (49.6-56.8), categorizing them as difficult (Table 1).⁶ Finally, only 1 document indicated that participants in the placebo group might receive vaccine. Even then, the reference was oblique, and failed to specify the timeline or other details.

Table 2 provides 3 examples of lengthy, complex descriptions that could be simplified without compromising—and maybe improving—participant comprehension. We formulated a substitute informed consent document covering the same topics that had fewer than 3000 words, with a reading time of 12.3 minutes, a reading level between grades 7 and 8, and a readability score of 61.8, which is higher than the recommended HHS threshold (Table 1; eAppendix in the Supplement).

This quality improvement study found that informed consent documents for the phase III randomized clinical trials of COVID-19 vaccines did not adhere to the widespread view about how informed consent should occur and what constitutes a good informed consent document. All the documents were long, written at a high school reading level, and would be deemed difficult by HHS.⁶ Despite their length, most documents did not inform participants in the placebo group what would happen were the vaccines proven safe and effective.

Why are these vaccine trial consent documents so long and difficult to read? While it is impossible to say definitively, we posit 3 contributing factors. First, institutional review board members may not insist on shorter, more readable documents and/or may require additional material that they think, without any supporting data, will improve information transfer or promote trust. Second, the documents may be created by copying sections from previous documents, causing the documents to grow by accretion without any effort to make them more succinct. For instance, the privacy sections in all 4 informed consent documents contained irrelevant material, and none appeared to be customized to COVID-19 vaccine studies. Third, when drafting documents, legal teams may prioritize exhaustive details or liability mitigation over participants’ comprehension.

It is possible to develop a measurably better informed consent document, one that is shorter, more readable, and uses less complex language. This requires work and effort at editing. Thus, it may be more useful for researchers to hire an editor to collaborate on creating better documents rather than leaving the creation of informed consent documents to researchers, legal teams, or others whose expertise is not careful, succinct writing.

This study has some limitations. Evaluating the extent to which these informed consent documents effectively communicate to ensure participants’ comprehension of each of the myriad elements of informed consent was outside the scope of this study, and this is an area that warrants additional research. It is important to concede that our rewritten form also did not achieve a grade 6 reading level. The Flesch-Kincaid Grade Level assessment depends, in part, on the syllable count of words with many syllables and the number of those words. Necessary medical terms and consent-related words without shorter substitutes can inflate the reading grade level of these documents. Thus, the appropriateness of this metric to evaluate informed consent documents merits reevaluation, perhaps with a focus on more qualitative alternatives. Such an analysis was also outside the scope of this analysis. Furthermore, we only evaluated phase III informed consent forms to allow for a more uniform comparison. While informed consent is perhaps more vital to protecting participants considering entry into earlier, riskier trial stages, the mixture of combined and stand-alone early phase COVID-19 vaccine trials compelled us to focus exclusively on documents from phase III trials.

The findings of this quality improvement study suggest that participant informed consent has been compromised by lengthy, complex documents. Federal guidance has not been sufficient to create shorter, more comprehensible informed consent documents. Achieving a robust informed consent process will require a renewed commitment to documents that eliminate verbose and irrelevant material. To fulfill the goal of valid informed consent, organizations involved in clinical trials should channel the judiciousness of an editor in drafting future documents.

Accepted for Publication: March 10, 2021.

Published: April 28, 2021. doi:10.1001/jamanetworkopen.2021.10843

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Emanuel EJ et al. JAMA Network Open.

Corresponding Author: Ezekiel J. Emanuel, MD, PhD, Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania, 423 Guardian Dr, Philadelphia, PA 19104 (zemanuel@upenn.edu).

Author Contributions: Dr Emanuel and Mr Boyle had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: All authors.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: All authors.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Boyle.

Obtained funding: Emanuel.

Supervision: Emanuel.

Conflict of Interest Disclosures: Dr Emanuel reported receiving personal fees from Center for Neurodegenerative Disease Research, Genentech Oncology, Council of Insurance Agents and Brokers, America’s Health Insurance Plans, Montefiore Physician Leadership Academy, RAND Corporation, Medical Home Network, Healthcare Financial Management Association, The Ecumenical Center, American Academy of Optometry, Associação Nacional de Hospitais Privados, National Alliance of Healthcare Purchaser Coalitions, Optum Labs, Goldman Sachs, Massachusetts Association of Health, District of Columbia Hospital Association, Washington University, The Atlantic, McKay Lab, American Society for Surgery of the Hand, Association of American Medical Colleges, America’s Essential Hospitals, Johns Hopkins University, Shore Memorial Health System, National Resident Matching Program, Tulane University, Oregon Health & Science University, United Health Group, Blue Cross Blue Shield, Center for Global Development, and CBI, and serving as a partner at ReCovery Partners, Oak HC/FT, and Embedded Healthcare outside the submitted work. No other disclosures were reported.

Funding/Support: This research was partially funded by a gift from the Colton Family Foundation.

Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.