Parental Actionability of Educational Materials Regarding Laryngotracheal Reconstruction

Laryngotracheal reconstruction (LTR) can be a life-saving procedure for children with subglottic stenosis. Nevertheless, as many as 20% of patients experience some complications, such as scarring, infection, restenosis, or fistulization.¹ Rarer complications, such as anastomotic separation and laryngeal edema, can be deadly.¹ In such circumstances, when the medical decisions that parents make for their children have important consequences, parents must understand the information that they read to make the most informed decisions. In 2014, the Agency for Healthcare Research and Quality developed a validated instrument called the Patient Education Materials Assessment Tool (PEMAT) that determines the understandability and actionability of written health information.² We evaluated LTR-related patient educational materials by using the PEMAT.

This study was deemed to be exempt from approval by the institutional review board of Boston University School of Medicine. A Google search was performed using the search term laryngotracheal reconstruction. The first 100 results were screened for LTR patient educational materials. Exclusion criteria consisted of articles unrelated to LTR, articles not written for patients, and non-English articles. Three of us (K.R.K., A.G., and C.J.C.-S.L.) scored each article using the PEMAT. The PEMAT is divided into the domains of understandability and actionability.³ Both understandability and actionability are scored from 0 to 100, with higher scores denoting more understandable or actionable materials.² Reading difficulty was calculated using the Flesch-Kincaid Grade Level readability test, Gunning-Fog Index, Coleman-Liau Index, Automated Readability Index, and Simple Measure of Gobbledygook.⁴ Readability formulas are graded on a numerical scale, with scores reflecting the equivalent academic grade needed to understand a document (eg, 5 indicates a fifth grade reading level). Interrater reliability was assessed using the Fleiss κ statistic. We used the Spearman correlation (r value) to determine the association among readability, understandability, and actionability.

Thirty-five unique articles met inclusion criteria. Interrater agreement was substantial for understandability (κ = 0.72; 95% CI, 0.68-0.76) and actionability (κ = 0.76; 95% CI, 0.72-0.78). Median understandability score was 72.2 (range, 25.0-93.8) (Table). Most articles provided information in a logical sequence, used everyday language, and organized material into sections. Only a minority of articles provided visual cues, defined medical terms, or provided titles and/or captions. The median actionability score was 50.0 (range, 20.0-100.0). No articles provided instructions for calculations or explained how to use diagrams. We found a positive correlation between understandability and actionability (r = 0.58).

The median Flesch-Kincaid Grade Level score was 13.2 (range, 7.3-18.3); Gunning-Fog Index, 15.4 (range, 8.8-20.7); Coleman-Liau Index, 13.3 (range, 6.8-20.0); Automated Readability Index, 12.6 (range, 4.7-18.7); and Simple Measure of Gobbledygook, 14.5 (range, 9.4-18.4). We found a negative correlation between readability and understandability (r = −0.34) and between readability and actionability (r = −0.43).

Although the internet has become a popular source of health information, the quality of online patient education materials is often questionable. Previous studies evaluating written health information are limited by the validity of methods used.⁵ To overcome previous limitations, we used the validated PEMAT instrument.⁶

Results suggest that revisions may be warranted for current LTR information to benefit a larger readership. We found that understandability and actionability often did not meet minimum recommended levels (a score of 70)² necessary for patients to understand or act on the information they receive. Most LTR-related information was also too difficult for the average American adult to read.⁵ Articles with more difficult reading levels were harder to understand, which is consistent with results by Balakrishnan et al,⁵ who also found a negative correlation between reading difficulty and understandability. Correlation studies also demonstrated that articles with more difficult reading levels were less actionable and that patients are more likely to act on information they can understand. To our knowledge, this study is the first to assess these last 2 associations and further underscores the importance of understandable health information. An important limitation is that the PEMAT cannot assess the quality of information provided. For example, text can be understandable and actionable but still contain information that is biased or inaccurate.

We propose several ways to improve current LTR-related information. Providing visual cues, limiting medical jargon, and including summaries are important first steps to improve understandability. Writers should also address the reader directly, break down actions into individual steps, and provide tangible tools to help readers take action, all of which are important components of actionable health information.

Corresponding Author: Kevin Wong, BA, Boston University School of Medicine, 72 East Concord Street, Boston, MA 02118 (kevwong@bu.edu).

Accepted for Publication: April 30, 2017.

Published Online: July 6, 2017. doi:10.1001/jamaoto.2017.0942

Author Contributions: Mr Wong and Dr Levi had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Wong, Levi.

Acquisition, analysis, or interpretation of data: Wong, Keefe, Gilad, Chong-Sun Li.

Drafting of the manuscript: Wong.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Wong.

Administrative, technical, or material support: Keefe, Gilad.

Study supervision: Wong, Chong-Sun Li, Levi.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none were reported.