Documents were manually edited for high and low estimation of scores dependent on sentence length (for all readability measures except FORCAST). In the high estimate (shown), forms were altered so that all lists were treated as 1 sentence with items separated by a comma. In the low estimate (eFigure 1 in the Supplement), forms were edited so that all list items were treated as independent sentences separated by a period. Boxes represent the median and interquartile range (IQR) for each distribution, with the lower and upper box limits defined by the 25th and 75th percentile, respectively. The median is represented by the line across each box. The upper and lower whiskers extend to the greatest and lowest datum within 1.5 × IQR above and below the upper and lower quartile, respectively. Any data points outside of this range are defined as outliers and are displayed as dots. Maximum reported grade levels for included readability scores were up to grade 19 (equivalent to a doctoral degree level of education) for some scores. Reported grade levels higher than grade 12 represent collegiate levels of education and higher. Current recommendations from the National Cancer Institute and National Institutes of Health state that consent forms for patients should be at the eighth-grade readability level (blue horizontal dashed line) or lower.9,10 Current recommendations from the American Medical Association state that all written materials for patients should be at the sixth-grade readability level (red horizontal dashed line) or lower.8 DRP (GE) indicates Degrees of Reading Power (Grade Equivalent); FK, Flesch-Kincaid; FORCAST, Ford, Caylor, Sticht; GF, Gunning Fog; SMOG, Simple Measure of Gobbledygook.
Raygor estimates are considered invalid for text containing a high percentage of words with more than 6 characters and short sentences.
eMethods 1. Questions sent to departmental leaders
eMethods 2. Description of readability indices
eTable 1. Comparison of departments offering and not offering radiotherapy consent forms in the survey
eTable 2. Statistics describing estimated readability of cancer radiotherapy consent forms in U.S. academic centers by seven readability measures
eFigure 1. Readability of consent forms for cancer radiotherapy in U.S. academic centers as measured by seven readability indices, using paired high and low estimates for each index
eFigure 2. Word cloud of commonly used difficult words
eTable 3. Commonly used difficult words with recommended alternatives
Customize your JAMA Network experience by selecting one or more topics from the list below.
Perni S, Rooney MK, Horowitz DP, et al. Assessment of Use, Specificity, and Readability of Written Clinical Informed Consent Forms for Patients With Cancer Undergoing Radiotherapy. JAMA Oncol. 2019;5(8):e190260. doi:10.1001/jamaoncol.2019.0260
Are written informed consent documents for cancer radiotherapy used in US academic medical centers at appropriate readability levels to ensure patient comprehension?
In this nationwide survey study and readability analysis, only 9 (8%) of 113 cancer radiotherapy clinical consent forms met the most permissive national recommendation (eighth grade level) for patient materials. Moreover, consent forms contained an average of 7.2 common difficult words.
High readability grade levels and common use of difficult words in radiotherapy consent forms may make it difficult for patients to give truly informed consent; therefore, reevaluation and modification of radiotherapy consent forms on a national scale is warranted.
Appropriate informed consent processes are crucial to preservation of patient autonomy and shared decision making. Although half of patients with cancer receive radiotherapy, it is unknown whether current consent practices are comprehensible for patients.
To characterize use, specificity, and readability of clinical informed consent forms for radiotherapy, hypothesizing that forms would be higher than the recommended sixth- to eighth-grade readability level.
Design, Setting, and Participants
This nationwide cross-sectional survey study and readability analysis was conducted from 2016 to 2018 and included 89 academic radiation oncology departments that were part of the 2016 Electronic Residency Application Service. Department leaders (clinical directors, chairs, and personal contacts of study authors) at academic radiation oncology departments were contacted via email.
Main Outcomes and Measures
Readability levels were measured by 7 validated readability indices, including the Ford, Caylor, Sticht (FORCAST) index for nonnarrative texts. Difficult words were identified using The Living Word Vocabulary, which describes the readability grade levels of 40 000 common words.
Of 89 departments, 67 (75%) responded to questions and 57 (64%) provided 113 forms for analysis. Departments providing forms did not differ substantially from others in terms of region, residency size, research output, rural vs urban location, or public vs private institution status. All departments obtained patient written informed consent before radiotherapy; 38 (57%) used body site–specific forms. Using the most conservative (low-score) estimate, mean form readability ranged from grade level 10.6 to 14.2. By 7 distinct indices, only 9 (8%) of 113 forms met the recommended eighth-grade readability level, and 4 (4%) forms met a sixth-grade level. Not a single form met either recommendation based on the FORCAST index. Forms used an average of 7.2 difficult words. Body site–specific forms had considerably better readability than general consent forms.
Conclusions and Relevance
This nationwide study of informed consent practices for cancer treatment with radiotherapy demonstrates that while all US academic radiotherapy departments use written consent forms, it is rare for templates to meet the recommended readability levels for patient materials. These data suggest the need for reevaluation and modification of the approach to radiotherapy consent, ideally with guidance and templates designed by national professional organizations.
Appropriate informed consent procedures are crucial to preservation of patient autonomy and facilitation of shared decision making. According to the National Academy of Medicine, “regulations that govern the attainment of informed consent…are crucial to ensuring that medical care and research are conducted in an ethical manner and with the utmost respect for individual preferences and dignity.”1 However, obtaining truly informed consent is a challenge in many medical settings, including cancer treatment, where the landscape of therapeutic options is increasingly complex and shifting. Informed consent for radiotherapy may be particularly difficult given its technical and abstract nature.2 Despite the associated risks, radiotherapy is received by about half of patients with cancer,3,4 and studies have shown that patients desire extensive information about their treatment and the associated adverse effects.5 Recent research suggests that nearly half of patients initiating radiotherapy have heard frightening stories, heightening the need for optimal communication during the consent process.6
Written consent forms standardize discussions and provide patients with a written document to which they can refer. Readability (the education level necessary for comprehension) of consent forms can thus affect what patients understand about their upcoming treatment. Readability standards have typically focused on research consent forms, with US standards typically ranging from grades 5 to 10.7 The American Medical Association recommends a sixth-grade readability level, whereas the National Institutes of Health recommend a grade 7 to 8 level.8,9 The National Cancer Institute (NCI) recommends an eighth-grade level, which reflects the readability level that the average US citizen comprehends.10,11 However, the average Medicaid enrollee reads at a fifth-grade level.12
Although the American College of Radiology, the American College of Radiation Oncology, and the American Society for Radiation Oncology all recommend written informed consent for patients with cancer undergoing radiotherapy, none of these guidelines discuss form readability.13-15 To our knowledge, only 1 study has evaluated the readability of radiotherapy consent forms; a small multimodal analysis of 3 forms for radiotherapy of cervical cancers found that they ranged in readability from grades 12.8 to 16.1.16
Given the importance of informed consent in cancer care and the lack of knowledge or standardized requirements for clinical consent forms for radiotherapy, we conducted a nationwide study to characterize informed consent practices across US academic departments. We analyzed the use and readability of consent forms, hypothesizing that readability would be above the NCI-recommended eighth-grade level.
After approval by the Columbia University Irving Medical Center and University of Michigan institutional review boards, we sent emails to department chairs or clinical directors of radiation oncology departments hosting all 88 Electronic Residency Application Service–listed radiation oncology training programs in 2016 (eMethods 1 in the Supplement). For the Harvard Radiation Oncology Program, the 2 associated departments (at Massachusetts General Hospital and Brigham & Women’s Hospital) were included separately. Department leaders were queried regarding use of consent forms and body site–specific consent forms and whether patients were able to take forms home prior to and after signature. General consent forms were requested, as were brain- and breast-specific forms, if used. When responses were not received, 2 investigators (R.J. and D.G.) emailed their personal contacts to follow-up. Characteristics of surveyed programs were gathered from an online residency navigator (www.doximity.com) to assess for response bias.
Readability analysis was conducted using Readability Studio 2012 (Oleander Software). Analyzed indices included Degrees of Reading Power (DRP) and grade equivalent (GE) test17; Flesch-Kincaid (FK) readability test18; Ford, Caylor, Sticht (FORCAST) index19; Fry score20; Gunning Fog (GF) index 21; Raygor estimate22; and Simple Measure Of Gobbledygook (SMOG) grade.23 These are commonly used and well-validated measures of readability that report grade-level equivalents.24 They are derived using parameters such as average sentence or word length, or difficult and/or uncommon words (eMethods 2 in the Supplement).
Most scores are calculated based on sentence length and are not designed to analyze nonnarrative texts (eg, bulleted lists). The FORCAST index is the only included metric that does not account for sentence length and counts the number of monosyllabic words used. Most forms in this investigation, however, used some type of nonnarrative text. To address this limitation, all forms were individually edited to create high- and low-score estimates of readability. In the high-score estimate analysis, all forms were edited and lists were treated as 1 sentence, separating each item with a comma and leading to higher average sentence length and readability level. In contrast, in the low-score estimate analysis, forms were edited so that each list item read as an independent sentence. Figure 1 was generated using R 3.3.3 (R Foundation) and Figure 2 was generated using Readability Studio.
All words with 3 syllables or more were extracted using Readability Studio’s difficult-word analysis function. Different tenses of the same word were combined. Individual word difficulty was assessed according to The Living Word Vocabulary, a text containing 40 000 unique words with associated readability grade levels.25 Items are reported as word meaning, grade level, and the percentage of readers at that grade level who understand the word’s meaning. Criteria for inclusion as a commonly used difficult word were use 1 or more times in 3 or more forms and no listing in The Living Word Vocabulary or reported as a 12th-grade level or higher, which is a substantial deviation from national recommendations. Unlisted words were included in the assessment because technical or medical terms were not included in The Living Word Vocabulary, which was intended to reflect vernacular language. Therefore, such words are more likely to be difficult for a general audience.
A word cloud representing commonly used difficult words was generated with Wordle.net (Jonathan Feinberg and IBM Corporation). Word clouds visually depict relative word frequency or importance. Herein, word size was determined according to a weighting factor given by the following equation: weighting factor = difficulty index × number of forms using word.
The difficulty index was determined by The Living Word Vocabulary reported grade level (not listed = 4, 16th grade = 3, 13th grade = 2, 12th grade = 1). The weighting factor was set at a maximum of 25 to prevent the most heavily weighted words from obscuring results.
After comparing characteristics of responding and nonresponding departments using 1-way t testing for continuous variables and χ2 testing for categorical variables, we generated a Raygor distribution for individual forms and summary statistics with distributions of readability for all forms in aggregate. Given recommendations for sixth- or eighth-grade readability levels, we evaluated whether forms met those levels as measured by all included readability indices, reporting findings individually by score using high and low estimates. Additionally, we evaluated whether each form met each recommendation using the most permissive index for that form, again describing findings using high and low estimates separately. Analysis of variance (ANOVA) was performed using Excel (Microsoft Corporation) to assess differences in readability across form types (breast-specific, brain-specific, general radiotherapy, and general medical) for each readability index, using low estimates of readability. A post hoc Tukey test was performed for ANOVA results with a significance level P < .05.
Of 89 surveyed departments, 67 (75%) responded to survey questions. All used written consent forms; 64 (96%) used a general form and 3 (4%) exclusively used site-specific forms. Thirty-eight departments (57%) had brain-specific and/or breast-specific forms. A total of 57 departments (64%) provided 113 unique forms, including 32 breast-specific, 34 brain-specific, 37 general radiotherapy, and 10 general medical consent forms used for all hospital procedures. Survey responders who offered forms were not considerably different from nonresponders in terms of department size, location, research output, urban vs rural setting, and public vs private institution status (eTable 1 in the Supplement). Only 2 (3%) departments reported routinely providing patients with forms to take home prior to provision of consent, whereas 30 (45%) indicated that they routinely provide patients with forms to take home following provision of consent. Other departments indicated that they would typically provide forms upon request.
High-estimate readability distributions, a Raygor distribution for individual forms, and summary statistics are shown in Figure 1, Figure 2, and eTable 2 in the Supplement, respectively. Paired high- and low-estimate distributions for each index are available in eFigure 1 in the Supplement. Regardless of index, most consent forms had grade levels well above national recommendations. On low-score estimate analysis, only 4 (4%) of 113 forms were at a sixth-grade level and 9 (8%) of 113 were at an eighth-grade level as measured by the most permissive score for each form. On high-score estimate analysis, no forms had a sixth-grade readability level and only 3 of 113 (3%) had an eighth-grade readability level. Mean readability grade levels for different scores ranged from 12.3 to 15.2 in the high-score estimate and from 10.6 to 14.2 in the low-score estimate. The mean FORCAST readability grade level was 11.5 (range, 10.3 to 13.6). When comparing readability levels between types of forms (body site–specific forms and general consent forms), site-specific forms were found to have lower readability levels (ie, lower grade levels) than general forms by most readability measures (Table).
Of the 107 words that met the inclusion criteria as commonly used difficult words (eFigure 2 and eTable 3 in the Supplement) based on the entries in The Living Word Vocabulary,2 45 words (42%) were at the 12th-grade readability level, 16 (15%) were at the 13th-grade level, 9 (8%) were at the 16th-grade, and 37 (35%) were unlisted. eFigure 2 in the Supplement visually depicts these difficult words in a frequency- and difficulty-weighted word cloud. The most frequently cited words were alternative(-ate), oncologist(-y), simulation, attending (physician), irradiated(-ion), (contra)indicated(-ions), intervention(-al), and recurrence(-t), each used in at least 15% of forms. Forms included an average of 7.2 (range, 0-29) difficult words. eTable 3 in the Supplement lists recommendations for alternatives to difficult words.
This nationwide study reveals that although all responding departments adhere to national recommendations to obtain written consent from patients with cancer prior to initiating radiotherapy, most fail to provide consent forms at or near the NCI-recommended eighth-grade readability level for research consent forms (Figure 1 and eTable 2 and eFigure 1 in the Supplement). Even using low-score estimates of readability, the highest proportion of forms meeting the most permissive recommendation of eighth-grade readability level was only 8% (n = 9 of 113), as measured by the lowest score for each form. Furthermore, the FORCAST score analysis, the only included index designed for nonnarrative texts, revealed that no form met readability recommendations.
Informed consent is a requirement for ethical medical practice26 and is widely accepted as a basic imperative applying to both research and clinical practice. Historically, unconscionable medical experimentation by the Nazis precipitated the Nuremberg Code and influenced subsequent documents such as the Declaration of Helsinki and Belmont Report that outline principles of ethical conduct in human research. As a result, informed consent in research is well studied and highly regulated. Routine clinical interventions are not experimental but still require informed consent because there may be treatment risks that outweigh benefits for certain patients. It is widespread practice to obtain written patient consent for clinical procedures, but clinical consent forms are much less regulated and studied than research forms.27 No clear guidelines exist regarding the readability of clinical informed consent forms for patients with cancer, and we know of no prior studies of the effectiveness of written informed consent processes for radiotherapy in the United States.
The available data on written informed consent processes for radiotherapy are extremely limited. A 2010 Canadian study found that 41% of respondents (12 centers) did not obtain written consent for radiotherapy; only 2 centers used body site–specific forms and explained risks.28 A similar Australian study showed that half of practicing radiation oncologists worked in departments without policies advising consent form use.29 In a 1997 European survey, only 28 of 97 (29%) centers reported using written forms.30 None of these studies examined readability.
In contrast, all responding departments in the present study used consent forms for routine clinical practice. However, 15% (n = 10 of 67) used general hospital consent forms typically designed for procedures that make no mention of radiotherapy or its adverse effects. Departments rarely give patients a form to take home prior to signing, and fewer than half routinely give patients a copy to take home after signing.
Our results indicate a substantial need for large-scale improvement of clinical consent practices for radiotherapy in US academic medical centers. These findings should also prompt similar evaluation of informed consent procedures in other clinical settings. In the 2 studies of readability of clinical consent forms for surgeries/procedures in the United States, average readability grade levels were 15 in Rhode Island hospitals31 and 12.6 in a 1998 nationwide study.32 Institutional review board–approved research consent forms for oncology trials have an average readability grade level of 10.3, but there are no studies of clinical consent form use and readability for chemotherapy or oncologic procedures.33
Effective informed consent is challenging for highly technical therapies such as radiotherapy. Patients may find it difficult to recall verbal discussions; written forms may help them understand or remember discussions of risks and benefits of treatments.34 In a meta-analysis of 21 randomized clinical trials, 16 revealed that written consent forms for specific procedures or anesthesia resulted in improved patient comprehension.35 Although lower readability scores may not necessarily translate into better patient understanding, improving forms likely removes an important obstacle. Consent conversations may be more understandable than their written counterparts but also have a higher likelihood of omitting critical elements such as toxic effects.36 Consent forms can thus be used as a decision aid and reference for patients, as well as a tool to standardize consent conversations.
Designing consent forms that optimize patient autonomy through comprehension of risks and benefits is a crucial albeit challenging first step in shared decision making. Shared decision making seeks to incorporate patient preferences and values into treatment decisions in situations with inconclusive clinical evidence, a close benefit-to-harm ratio, or considerable variation in patients’ opinions about desirability of choices. Shared decision making is often employed in cancer settings, where consideration of quality of life is important and many decisions are preference sensitive.37 Without an accurate understanding of treatment choices and risks, shared decision making is not possible. Optimal consent procedures are thus necessary (although not sufficient) for shared decision making.
Several strategies emerged from this study to guide efforts to improve the extent to which written informed consent forms could truly enhance patient understanding. First, specificity regarding treatment site was associated with improved readability, and thus efforts to be as specific as possible in consent form design might translate into better patient comprehension. However, even when being specific, many technical or medical terms were used; 24 (21%) of 113 forms used the word simulation. According to The Living Word Vocabulary, only 61% of individuals at the 12th-grade reading level understand the word’s meaning. Difficult words and suggested modifications identified in this study (eTable 3 in the Supplement) should be used as a resource for the development or modification of radiotherapy consent forms.
Additionally, some forms included long lists of adverse effects, which may provoke anxiety. The use of bipolar structures (eg, listing common vs rare or short-term vs long-term adverse effects) has been shown to increase patient-reported understanding and satisfaction with how information is relayed to patients with cancer.38
Ensuring efficient provision of compassionate, tailored, contextualized education for a diverse patient population can challenge even experienced clinicians. Patients receiving radiotherapy may differ in socioeconomic and cultural characteristics, which are proven determinants of health literacy and may affect patients’ abilities to comprehend written forms.39 Patients may have different preferences or abilities in terms of the level of information required to give informed consent, such as varying desires to know about treatment risks.5,40 The consent process may be even more difficult for non-English speakers, those requiring proxied consent, or those receiving emergent treatment.41 Careful consideration of patient characteristics is thus paramount in the development of clinical consent forms. Departments might consider having different consent forms tailored to patients’ reading levels, such as having easy-to-read versions of forms. Principles of design, such as organizing content logically, using sufficiently large typeface, and incorporating diagrams may also be useful in optimizing forms. A guide to document design is available from the Centers for Disease Control.42
Despite its strengths of drawing from a national sample and rigorous analyses using multiple formal readability standards, this study has limitations. First, although we show that consent forms have high readability levels, we do not correlate these forms with patient preferences or understanding. Further investigation of patient preferences regarding radiotherapy-specific consent forms, efficacy of consent interventions, and the association of patient demographic and socioeconomic factors on form readability would be valuable adjuncts for contextualizing the present study. Additionally, although no surveyed institutions reported using multimedia or digitized consent tools, these novel methods may improve informed consent for patients with a low level of health literacy who are entering research trials.43,44 Therefore, the development, evaluation, and dissemination of innovative consent methods (eg, video- or multimedia-assisted tools) in conjunction with the improvement of written forms may assist diverse patient populations in overcoming barriers to informed consent.
Second, the response rate was 75% (n = 67 of 89 departments), with 64% of departments (n = 57 of 89) providing forms for analysis. Although this rate is high for a national survey,45,46 it is possible that the results were affected by response bias. However, there were no marked differences in the characteristics of departments providing forms and those that did not, rendering marked response bias less likely. Third, the nonnarrative structure of many of the forms makes most available readability tests less accurate. To address this, forms were edited to create high- and low-score readability estimates. The FORCAST score, which is independent of narrative structure, was also used but may not effectively capture readability alone. Despite this limitation, multiple metrics had consistent findings of high readability grade levels even after editing the forms in ways deliberately intended to improve readability, which suggests that the findings are not an artifact of a particular measurement. Finally, The Living Word Vocabulary, which was used as the measure of individual word difficulty, was published in 1979 and omits certain now common words, such as digital. Nevertheless, this approach provides a systematic methodology by which to identify difficult words and remains the most comprehensive source for word grade levels; moreover, a word cloud is provided so that the reader may judge independently the ease of comprehension of words identified in the analysis.
Effective and ethical informed consent practices are crucial to protection of patient autonomy and shared decision making. This nationwide study of informed consent practices for cancer treatment with radiotherapy demonstrates that although all US academic radiotherapy departments use written informed consent forms, it is rare for them to meet the recommended readability levels for patient materials. Difficulty comprehending consent forms may present a considerable obstacle to patients who are attempting to make informed, challenging decisions regarding treatment for cancer. These data suggest a need for reevaluation and modification of many current cancer radiotherapy consent documents using simple strategies, as well as the need for further research to evaluate consent processes in other settings—ideally with guidance from and templates designed by national professional organizations.
Accepted for Publication: January 16, 2019.
Corresponding Author: Andrew J. Einstein, MD, PhD, Columbia University Irving Medical Center, 622 W 168th St, PH 10-203B, New York, NY 10032 (firstname.lastname@example.org).
Published Online: May 2, 2019. doi:10.1001/jamaoncol.2019.0260
Author Contributions: Drs Jagsi, Horowitz, and Einstein had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis; they played complementary roles as senior authors and mentors to the first authors. Dr Perni and Mr Rooney contributed equally as co-first authors.
Study concept and design: Perni, Horowitz, Golden, McCall, Einstein, Jagsi.
Acquisition, analysis, or interpretation of data: Perni, Rooney, Horowitz, Golden, Einstein, Jagsi.
Drafting of the manuscript: Perni, Rooney.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Perni, Rooney.
Administrative, technical, or material support: Perni, Horowitz, Einstein.
Study supervision: Horowitz, Golden, McCall, Einstein, Jagsi.
Conflict of Interest Disclosures: Dr Horowitz reports receiving consulting fees and travel reimbursement from Carl Zeiss and consulting fees from Champions Oncology. Dr Einstein reports receiving grant funding for unrelated research from the National Heart Lung, and Blood Institute, the National Cancer Institute, the International Atomic Energy Agency, Canon Medical Systems, and Roche Diagnostics and consulting fees from GE Healthcare. Dr Jagsi reports receiving grant funding for unrelated research from the National Cancer Institute, Doris Duke Foundation, Greenwall Foundation, Komen Foundation, and Blue Cross Blue Shield of Michigan/Michigan Radiation Oncology Quality Consortium; has received consulting fees from Amgen and Vizient; and has stock options in Equity Quotient. No other disclosures were reported.
Additional Contributions: The authors gratefully acknowledge all of the survey respondents. They received no compensation for their contributions to the study.