Evaluation of the Informational Content and Readability of US Lung Cancer Screening Program Websites

IMPORTANCE The internet is an important source of medical information for many patients and may have a key role in the education of patients about lung cancer screening (LCS). Although most LCS programs in the United States have informational websites, the accuracy, completeness, and readability of these websites have not previously been studied. OBJECTIVE To evaluate the informational content and readability of US LCS program websites. DESIGN, SETTING, AND PARTICIPANTS This cross-sectional study assessed US LCS program websites identified on September 15, 2018. A standardized checklist was used to assess key informational content of each website, and text was analyzed for reading level, word count, and reading time. Links to US websites of national advocacy organizations with LCS program content were tabulated. All functional LCS program websites in Google internet search engine results using the search terms lung cancer screening , low-dose CT screening , and lung screening were included in the analysis. eligibility


Introduction
In the United States, lung cancer has the highest cancer-specific mortality rate. The National Lung Screening Trial (NLST) showed a 20% reduction of lung cancer-specific mortality with low-dose computed tomography (LDCT) after 3 annual screenings compared with a chest radiography control arm. 1 After publication of the NLST results, the US Preventive Services Task Force and the Centers for Medicare & Medicaid Services (CMS) soon followed with recommendations for LDCT lung cancer screening (LCS). 2 This led health care plans offered under the Affordable Care Act's Health Insurance Marketplace, as well as most private plans, to cover certain preventive care services, such as LDCT LCS, without patient cost sharing. 3 Despite the proven benefits of LCS among eligible patients, the insurance coverage, and the increasing number of LCS facilities across the United States, the rate of LCS participation remains low. It is estimated that only 1.9% of up to 8 million screening-eligible US adults have been screened. 4 Accessible and comprehensive patient information on eligibility, benefits, risks, costs, and logistics of LCS may be important for facilitating understanding of this still underused service.
Patients eligible for LCS may have questions and may experience confusion and anxiety about the screening process. [5][6][7] Lung cancer screening may prompt follow-up diagnostic examinations, procedures, visits, and therapies that are often subject to patient cost sharing in the form of deductibles and copayments, in addition to the potential costs of travel, missed work, and similar expenditures. 8 Prior evidence has shown that medical information for patients is typically written at a reading level higher than the grade 8 level of the average US adult. 9 Many current and prior smokers may have cognitive and health literacy challenges, underscoring the need for accurate information written at an accessible level. 10,11 Shared decision-making, in which clinicians discuss eligibility, benefits, and risks of LCS with patients as mandated by the CMS, is intended to address questions and concerns, but many health care professionals have found it difficult to cover LCS optimally within the time constraints of a patient visit. [12][13][14][15] Many patients turn to the internet for more information. Most adults in the United States have used the internet to gather health information, 16 and internet-based resources have been shown to increase LCS participation among patients. 17 Numerous national advocacy organizations have created websites with LCS information, but the websites of LCS programs may be the first or primary source of information for many patients. The objective of this study was to evaluate the informational content and readability of US LCS program websites.

Website Selection
In this cross-sectional study, all functional LCS program websites were identified with a Google internet search using the search terms lung cancer screening, low-dose CT screening, and lung screening on September 15, 2018. We used a Mozilla Firefox browser in incognito mode connected to a virtual private network server to prevent any customization to the search algorithm that may result from location, cookies, or user account information. 18

Content Analysis
Included websites were randomly assigned for analysis to 1 of 9 thoracic radiologists (all authors) participating in an LCS program. Radiologists reviewed text, images, videos, and other file attachments on the main landing pages of LCS programs and linked pages with a standardized JAMA Network Open | Imaging LCS program's institution were not included in the content analysis. The lead and senior authors (S.M.G. and B.P.L., respectively) confirmed the accuracy of entries after the primary content assessment; any discrepancies were resolved by consensus between the 2, and any missing data were added after additional website review.

Classification of Practice Type and US Geographic Region
The practice setting of each LCS program website was classified as academic, community, hybrid, or other. The academic classification was used when the institution was a teaching hospital affiliated with a medical school. Community practice was used for community hospitals not associated with an academic institution. Hybrid status was assigned when the institution was a health network or system with private, community, and sometimes academic components. Programs classified as other did not fit any of these categories and included private radiology practices not primarily affiliated with an institution. The US geographic region was determined by zip codes.

Readability Analysis
Text was manually copied into Microsoft Word (Microsoft Corp) and cleaned of formatting and of content, including figures, captions, hyperlinks, advertisements, and addresses. Readability was assessed with a commercial web-based suite of tools (readable.com). The average grade-level score comprised the mean of 5 widely used and validated readability indexes. [22][23][24][25][26] The Flesch-Kincaid score is based on the average number of words per sentence and the mean number of syllables per word. 22 The Gunning-Fog index tabulates reading level with a formula using the number of sentences, the number of words, and the number of words with 3 syllables or more. 23 The Coleman-Liau formula requires the number of characters and sentences. 24 The Simple Measure of Gobbledygook (SMOG) selects 10 sentences from the beginning, middle, and end of a text and uses the number of words with 3 syllables or more. 25 The Automated Readability Index represents difficulty through a ratio of the number of letters in a word to the number of words in a sentence. 26

Statistical Analysis
Microsoft Excel (version 16.27), including the XLSTAT add-in (version 2019.1.1; Addinsoft), was used for data processing and statistics. Descriptive statistics were used to characterize the informational content and readability of each LCS program website. The practice type of the associated institution, the US geographic region, and the presence of material regarding LCS eligibility criteria, benefits, risks, costs, and logistics were summarized. The medians and SDs of website readability scores, word count, and reading time were calculated. Scatterplots and boxplots were generated to show variability in website readability scores and word counts.

Results
In total, 257 unique US LCS program websites of 269 results returned by the Google internet search were included in the analysis. The other 12 search results were links to nonexistent web pages, duplicates, or websites without LCS-related information. The practice types and US geographic regions of these LCS programs are listed in Table 1.

Enrollment Criteria
Information about having a smoking history of 30 or more pack-years was listed on 241 of 257 websites (94%), and data on being an active smoker or having quit within 15 years was detailed on 231 websites (90%) ( Table 2). There was wide variability regarding reported eligibility in age criteria, with ages 55 to 74, 55 to 77, and 55 to 80 years cited on 14% (n = 35), 42% (n = 108), and 16% (n = 42) of websites, respectively; 19% (n = 48) of websites mentioned more than 1 age range. Lack of signs or symptoms of cancer was mentioned on 120 websites (47%), and willingness and/or ability to undergo curative treatment was mentioned by only 18 websites (7%).

Screening Logistics
Frequency of LDCT examinations was mentioned on 108 of 257 websites (42%) and was correctly described as annual on only 87 websites (34%) ( Table 2) Table 2.

Costs of Screening
The subject of patient cost was mentioned on 192 of 257 websites (75%) ( Table 2). Medicare or private insurance was listed as a mode of coverage by 131 websites (51%). A specific dollar amount was provided by only 31 websites (12%), whereas cost without a specific amount was mentioned on only 21 websites (8%). Nine websites (4%) indicated that LCS was free.

Benefits and Risks of Screening
In terms of major LCS benefits, the low radiation dose of LDCT was mentioned on 239 of 257 websites (93%), and the ease of LDCT was mentioned on 164 websites (64%) (

Readability, Word Count, and Reading Time
The median reading level of all websites was grade 10 (interquartile range [IQR], 9-11) (Figure, A). The minimum reading level was grade 5, and the maximum reading level was grade 16. Only 4 of 257 websites had reading levels at or below the grade 6 level recommended by the American Medical Association for patient materials. 27 An additional 41 websites were written at or below the grade 7 to grade 8 level recommended by the National Institutes of Health, for a total of 45 websites meeting this less stringent guideline. 9 The word count ranged from 73 to 4410 (median, 571; IQR, 328-909)   (7) Lack of signs or symptoms of cancer 120 (47) Willingness and/or ability to undergo curative treatment 18 (7) Logistics of Screening Smoking cessation mentioned 143 (56) Website link or phone number for smoking cessation 94 (37) Frequency of LDCT examinations mentioned 108 (42) Annual 87 (34) Annual follow-up duration of 3 y 9 (4) Annual until quit smoking ≥15 y or exceed age criterion 8 (3) Follow-up not otherwise specified 2 (1)  (10) Out-of-pocket costs 17 (7) Abbreviations: LDCT, low-dose computed tomography. (Figure, B). The reading time in minutes ranged from 0.3 to 19.6 (median, 2.5; IQR, 1.5-4.0). A description of differences across US geographic regions is summarized in Table 4.

Discussion
Because most US adults use the internet to find health information, 16 the accuracy of health care websites is of paramount importance. Of those who search online for information on LCS, 77% begin with a search engine 16 and likely find the websites evaluated in this study. In our comprehensive analysis of the LCS program websites, we found inconsistencies, incompleteness, and a reading level  Reading levels were assessed as the mean of 5 validated readability scores (Flesch-Kincaid score, 22 Gunning-Fog index, 23  above that of the average US adult (grade 8 level). 9 Unlike other studies 11,28 investigating the reading level of online LCS information, the present study evaluated the content of LCS program websites, which may serve as a primary source of information for many LCS-eligible persons. The LCS program websites offer an opportunity for comprehensive coverage of enrollment criteria, logistics, costs, benefits, and risks of screening. Such information can supplement the shared decision-making process that occurs during a patient visit, which is often constrained by time and by competing health concerns. Websites can also link to national organization websites that have peer-reviewed decisionmaking tools.

Enrollment Criteria and Screening Logistics
Prior investigations have found low rates of familiarity with LCS eligibility criteria among primary care clinicians. In a recent study, 29 only 31% of clinicians knew the correct eligibility criteria for LCS. Our data demonstrated that LCS program websites provide conflicting information that may confuse patients and their health care clinicians. For example, the upper age limit for LCS was sometimes listed as age 80 years, as recommended by the US Preventive Services Task Force 21 ; age 77 years, the upper age limit covered by CMS 2 ; and age 74 years, which was the upper age limit in the NLST and is also listed on websites by the American Cancer Society, 30  In addition, Cataldo 34 found that high-risk smokers are interested in quitting; therefore, the information in the setting of LCS should be made available to them at every opportunity.

Costs of Screening
In a survey of 338 LCS-eligible persons in 2015, a total of 78.4% (265 of 338) indicated interest in information about the cost of LCS. 34 As part of the Affordable Care Act, LCS is covered by the CMS

Readability, Word Count, and Reading Time
The accessibility of health information has been an ongoing concern for the medical community, with many patient materials written at a higher reading level than that of the average American adult.
Although the American Medical Association recommends a grade 5 or grade 6 reading level for patient medical information, 27 only 4 of 257 LCS program websites met this requirement. Forty-five of 257 websites met the grade 7 or grade 8 level recommended by the National Institutes of Health. 9 Most US adults read at a grade 8 or grade 9 reading level, and 20% read at a grade 5 level or lower, but health care materials are often written at a grade 10 level or higher. 48 Of the websites we studied, 138 of 257 (54%) were written at a grade 10 level and higher.
Several prior studies 9,11,18,28 have assessed the readability of health care information for patients, a handful of which evaluated LCS patient information available on the internet. 11,28 However, these studies were limited to a smaller subset of patient educational materials rather than JAMA Network Open | Imaging . Although readability is an important consideration for assessing accessibility of website text, word count and reading time may alter overall effectiveness by influencing reader engagement and retention. In our study, word count ranged from 73 to 4410, and reading time ranged from 0.3 to 19.6 minutes. We are not aware of formal recommendations for optimal length of online health information, but eye tracking studies of websites from a variety of industries by the Nielsen Norman Group 49 have shown that users rarely read beyond the third screenful of information and would rather scroll than click through a series of short web pages. Our study provides information on the total reading time of LCS program websites, but future observational studies examining the optimal content, length, and structure of LCS program websites may be warranted.

Limitations
This study has limitations. Readability tests provide an estimate of the understandability of text; however, they do not account for the content conveyed by videos, pictures, and schematics that may be included on websites. Content review can be subject to omissions and interpretive errors, but we attempted to minimize these by using a structured template for scoring websites, by randomly distributing websites to subspecialists with experience in LCS, and by reviewing results for accuracy and completeness. Although creating a comprehensive content checklist is challenging, we attempted to score the most substantial topics as highlighted in the recent LCS literature. Finally, although our study may not have included every US LCS program website, we used websites found in the results of the most widely used search engine.

Conclusions
This study found that information provided on the websites of LCS programs varies widely. Reading