Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
July 2001

The Readability of Pediatric Patient Education Materials on the World Wide Web

Author Affiliations

From the Department of Pediatrics, Children's Hospital of Iowa, Iowa City.


Copyright 2001 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.2001

Arch Pediatr Adolesc Med. 2001;155(7):807-812. doi:10.1001/archpedi.155.7.807

Background  Literacy is a national and international problem. Studies have shown the readability of adult and pediatric patient education materials to be too high for average adults. Materials should be written at the 8th-grade level or lower.

Objective  To determine the general readability of pediatric patient education materials designed for adults on the World Wide Web (WWW).

Materials and Methods ( is a digital library serving the medical information needs of pediatric health care providers, patients, and families. Documents from 100 different authoritative Web sites designed for laypersons were evaluated using a built-in computer software readability formula (Flesch Reading Ease and Flesch-Kincaid reading levels) and hand calculation methods (Fry Formula and SMOG methods). Analysis of variance and paired t tests determined significance.

Results  Eighty-nine documents constituted the final sample; they covered a wide spectrum of pediatric topics. The overall Flesch Reading Ease score was 57.0. The overall mean Fry Formula was 12.0 (12th grade, 0 months of schooling) and SMOG was 12.2. The overall Flesch-Kincaid grade level was significantly lower (P<.0001), at a mean of 7.1, when compared with the other 2 methods. All author and institution groups had an average reading level above 10.6 by the Fry Formula and SMOG methods.

Conclusions  Pediatric patient education materials on the WWW are not written at an appropriate reading level for the average adult. We propose that a practical reading level and how it was determined be included on all patient education materials on the WWW for general guidance in material selection. We discuss suggestions for improved readability of patient education materials.

LITERACY IS a national and international problem. Recent studies have shown that almost one quarter of the adult population in the United States and Canada are functionally illiterate (reading level below the fifth grade); an additional one quarter have poor reading and comprehension skills. Other countries' statistics are almost as bleak.1,2

In the United States, the average reading level is eighth to ninth grade. By comparison, the average newspaper is written at the 9th- to 12th-grade level.3 Studies have shown that literacy levels are approximately 3 to 5 grades below the highest grade of schooling completed.4 Patient education materials should be aimed at the eighth-grade level or below3; however, most patient education materials are still written at the 10th-grade level or higher.511 Studies of pediatric patient education materials in outpatient clinics, emergency departments, and Women, Infant, and Children program clinics have found that the reading levels were consistently higher than recommended.2,1215 A previous study of the readability of general patient education materials on the World Wide Web (WWW) revealed the mean reading level at almost the 10th-grade level.16

Pediatricians and other pediatric health care providers often use written patient education materials in their practices. Material sources are varied, including those written or modified by pediatricians or their staff members. Therefore, pediatricians not only need to be aware of the materials' accuracy, but also should have some knowledge about the readability of the materials to ensure that accurate information can be comprehended. The goal of this study was to determine the general readability of pediatric patient education materials designed for adults on the WWW.

MATERIALS AND METHODS ( is a digital library serving the medical information needs of pediatric health care providers and patients. As of October 2000, it contained 1994 hyperlinks to 439 authoritative Web sites. The digital library links to pediatric health information on 390 common problems faced by pediatric health care providers and patients. Although the digital library does not provide the health information itself, it links to other authoritative information using the following criteria of displaying the author(s), institution(s), currency of the information, sponsor(s), and references.1724 The criteria did not include readability ( The Common Problems for Patients and Families page ( has information on 204 common problems, with 1122 hyperlinks to 236 different Web sites. The linked content is written for adolescents and adults and not for younger children.

A researcher (P.K.) systematically evaluated every 10th link and its corresponding document on the Common Problems for Patients and Families page in March 2000. To obtain the broadest view of the patient education materials on the WWW, only 1 patient education document was evaluated from an individual Web site (eg, 1 document from the American Academy of Pediatrics, 1 document from the Centers for Disease Control and Prevention). If during the systematic evaluation, another link from a previously used Web site was chosen, this link was skipped over and the next 10th link was evaluated. Similarly, if the link led to a document that was not appropriate for this evaluation (eg, a table of contents), then the next 10th link was used instead. One hundred Web sites and their corresponding documents were evaluated. Each document's title and information was copied into Microsoft 98 (Microsoft Corp, Redmond, Wash). The documents' Flesch Reading Ease score and Flesch-Kincaid reading level were generated using the built-in software in Microsoft 98 and entered into a Microsoft Excel spreadsheet. The same documents were printed and readability levels were determined by hand calculation by a second researcher (J.J.-W.) using the Fry Formula and SMOG methods.25,26 A third researcher (D.M.D.) also hand calculated 10% of the documents and found the reading levels to be accurate. The Fry and SMOG readability levels were entered into the Microsoft Excel spreadsheet. Means were obtained for the Flesch Reading Ease scores only because its results are not directly comparable with the other 3 readability scores. Statistical evaluation of the Flesch-Kincaid, Fry, and SMOG reading levels, using analysis of variance and paired t tests, were calculated using the built-in statistical software in Microsoft Excel. A predetermined value of P<.05 was used for significance.

There are approximately 40 different readability formulas, all of which will give a reasonably accurate grade level (generally plus or minus 1 grade level with 68% confidence, an acceptable standard in the field).3 These 2 readability levels were chosen because they are built into Microsoft Word and can be easily obtained by users of this common software program. The Flesch Reading Ease score and Flesch-Kincaid reading level are 2 of the most widely used and validated systems for scoring readability.27,28 The Flesch Reading Ease ranges from 0 to 100, with a lower score being more difficult to read than a higher score. It uses sentence length and polysyllabic words to determine difficulty and does not give a grade level. It is a standard used by most of the insurance industry for consumer documents and contracts. A score of 70 or above is described as "easy" and is written at the grade school level. A score of 60 to 70 is described as "standard" and is written at approximately at the high school level. A score of 60 or below is described as "difficult" (Table 1). The Flesch-Kincaid score is only valid between US grades 3 and 12 and cannot differentiate information that is easier or more difficult than these levels. It uses mean sentence and word length to determine reading level. The Fry Formula is a hand-calculation method that is valid between the 1st- and the 17th-grade levels and is the method for patient education materials promoted by experts in the field.3 It uses average numbers of sentences and syllables in 100 word passages to determine the reading level. The SMOG is another hand calculation method used commonly for health information. It is valid from grades 3 through 19, but maybe less accurate below the sixth grade. The SMOG uses the number of polysyllabic words in 10 sentence passages to determine reading level. In general, improved readability has been shown for low numbers of words per sentence, characters per word, syllables per word, and percentage of passive sentences.3 Illustrations and tables also improve the readability.

Table 1. 
Image not available
Flesch Reading Ease Score Interpretation*

Data concerning a document's Internet domain, author(s), and institutional affiliation were also collected. A document's author may be an individual, group, or the institution itself. For example, a document written by Jane Doe, MD, for the XYZ Professional Society of America was coded into the "physician" author and "professional society" institution groups. If no specific individual(s) authored the document, the document was coded into "professional society" for both the author and institution groups.


One hundred documents from 100 different Web sites were evaluated. Eleven documents were not further evaluated because their Flesch-Kincaid reading level was below 3.0 or above 12.0; therefore, those reading levels were not valid. There were 5 documents that were below the 3.0 grade level and 6 documents that were above the 12.0 grade level. Eighty-nine documents composed the final sample. The documents that were evaluated covered a wide spectrum of pediatric topics (Table 2). The documents came from the domains of .org (n = 32), .edu (n = 20), .com (n = 19), .gov (n = 10), .net (n = 4), and international domains (n = 4). The majority of the documents specifically noted that they were designed for laypersons (n = 71). Some documents contained illustrations (n = 11) and tables (n = 8). No documents had their reading level noted.

Table 2. 
Image not available
List of Topics

The overall mean for the Flesch Reading Ease was 57.0 (described as "fairly difficult" [range, 52.1-65.1]). The overall means for Flesch-Kincaid reading level was 7.1 (range, 3.5-10.9); for the Fry Formula, 12.0 (range, 6.0-17.0); and for the SMOG method, 12.2 (range, 7.0-17.0). The Flesch-Kincaid reading level was statistically different from the Fry Formula and SMOG methods (P<.0001), and the Fry Formula and SMOG methods were not statistically different from one another (P = .44). Documents with illustrations and tables did not have substantively different mean reading levels (data not shown).

Table 3 shows the mean reading levels based on the author and the institution of the document. The Flesch Reading Ease scores for all groups were in the fairly difficult to standard range. The Flesch-Kincaid mean reading level was below the eighth-grade level for all author and institutional groups. In contrast, no author or institutional group had a mean reading level below 10.6 for the Fry Formula and SMOG methods. In general, the Flesch-Kincaid reading levels were 4 to 5 grade levels lower than for the Fry Formula and SMOG methods. The Flesch-Kincaid reading level was again statistically different from the Fry Formula and SMOG methods for each author and institutional group (all groups, P<.01). The Fry Formula and SMOG methods were not statistically different for each author and institutional group (P = .15 to P>.99)

Table 3. 
Image not available
Mean Reading Levels for Author and Institution Groups and Overall Sample

The goal of this study was to determine the general readability of pediatric patient education materials designed for adults on the WWW. This study is unfortunately consistent with previous studies, which showed that the readability of patient care information in both print and the WWW are written at too high a level for most adults. This has significant implications, as adults are the main group health care providers are educating on how to take care of themselves and their children. Using the Flesch Reading Ease, the patient education materials on the WWW were written at the fairly difficult level. By the Fry Formula and SMOG methods (the readability formulas recommended for patient education materials), the materials are written at the 12th-grade level, with no author or institutional group being less than the 10.6 grade level. The Flesch-Kincaid computer method did find the same pediatric education materials to be written at the 7.1 level overall, substantially below the other readability formulas. This is most likely because of some of the inherent problems using readability formulas, including those generated via computers, since each formula uses slightly different mathematical methods to calculate a readability level. Mailloux et al29 compared 4 software readability programs and found a variation of 1.3 grade levels between the programs, with Microsoft Word having the largest variance (approximately 3 grade levels).

Pediatricians and other pediatric health care providers have the opportunity to improve the readability of their patient education materials in the following ways: (1) Pediatricians should be aware that the readability of the materials they are giving parents is probably too high. (2) When these patient education materials are written or modified, the provider can use their well-honed professional skills with children to express the information using noncomplex language. (3) Computer formulas may be helpful during the writing and editing process to obtain an initial "ballpark" readability level. We suggest using the Flesch Reading Ease score, as it shows less variability and its interpretations are more consistent with the hand-calculation methods. We suggest that final documents should have the readability level reanalyzed using the Fry Formula or SMOG method, which are easily obtained through the WWW.30,31 (4) The readability level should be noted on the patient education materials for general guidance in material selection. This study found that none of the patient education materials had reading levels noted. An individual patient education document may still not be written at the appropriate level for an individual adult. The simple addition of a reading level on this material would make it easier for busy health care providers and patients to choose more appropriate patient education materials. Although some providers may be concerned that good readers will feel talked down to by instruction that has a low reading level, it has been shown that adults at all reading levels prefer and learn better with easy-to-read materials.3,32 Suitability of reading materials depends on many factors, but the reading level is usually the main criterion predicting patient comprehension of the material.3 Adapting the Suitability Assessment of Materials instrument3 and the Flesch Reading Ease scores, the following designations could be used: (1) easy reading (Flesch Reading Ease, >80 [<6th-grade level]); (2) adequate or standard reading (Flesch Reading Ease, 60-80 [6th- to 8th-grade level]); (3) and difficult reading (Flesch Reading Ease, <60 [≥9th-grade level]).

As standards develop for consumer health information on the WWW, we propose that a reading level and how it was determined also be included, again as a general guide to material selection. Writing patient education materials for print literature can be challenging and the WWW presents its own additional challenges. Many resources on improving the readability of patient education materials and designing materials for the WWW are available, including the tips described in Table 4.30,31,3335 The materials should use short sentences, be written in the active voice, and give examples when appropriate. The materials should also use large easy-to-read type and contain illustrations if appropriate. Definitions and glossaries help readers with new vocabulary. Pediatricians should remember that these patient education materials may be printed by users and therefore are dual-purpose materials (ie, both print and WWW). There are also methods of determining an individual's reading ability in the clinic setting, suitability of materials,3 and readability formulas for languages other than English36; however, these discussions are beyond the scope of this article.

Table 4. 
Image not available
Tips for Improved Readability3,29,30

This study is limited in that it only evaluated 100 documents on the WWW, which has an estimated 50 million pages of health-related information.37 We attempted to evaluate a wide range of pediatric information from numerous sources. Other limitations of the study are that the patient education materials were written in English only, and do not address the patient education needs of younger children. The researchers originally choose the hyperlinks in the digital library using criteria that did not include readability, but there still could be an unrecognized systematic bias. It is less likely that document selection bias occurred because only 1 document from a single Web site was selected and a variety of different types of Web sites were evaluated. The numbers in some of the author and institution groups are small and therefore may be subject to sampling error.

Finally, readability formulas themselves may underestimate the difficulty of medical information, because unfamiliar terminology increases the difficulty. The typical adult has little exposure to medical terminology; often the meaning has to be determined anew because of the lack of associations and references. Therefore, even short words increase the difficulty but are scored as easy to read in the formulas. The comprehension level needed to understand medical information might be even higher than that determined by the readability formulas used.3,16 Using definitions or glossaries can help this common problem.


Pediatric patient education materials on the WWW do not meet the guideline of being written below the eighth-grade level. We propose all patient education materials on the WWW also include a practical reading level and how it was determined to guide material selection. Computer-generated reading levels could be used as an initial measure of readability, but patient education materials should be reevaluated using another method.

Back to top
Article Information

Accepted for publication February 2, 2001.

Supported in part by the Robert Wood Johnson Foundation Generalist Faculty Scholars Grant, Princeton, NJ (Dr D'Alessandro).

We thank Leonard Doak, PE, and Cecilia Doak, MPH, for their helpful insights into readability formulas.

Corresponding author: Donna M. D'Alessandro, MD, Department of Pediatrics, Children's Hospital of Iowa, 200 Hawkins Dr, Iowa City, IA 52242-1009 (e-mail:

National Center for Education Statistics, Adult literacy levels.  Washington, DC US Dept of Education1999;Available at February 24, 2000
Chacon  DKissoon  NRich  S Education attainment level of caregivers versus readability level of written instructions in a pediatric emergency department. Pediatr Emerg Care. 1994;10144- 149Article
Doak  CCDoak  LGRoot  JH Teaching Patients With Low Literacy Skills. 2nd ed. Philadelphia, Pa JB Lippincott1996;
Lee  PP Why literacy matters: links between reading ability and health. Arch Ophthalmol. 1999;117100- 103Article
Albright  Jde Guzman  CAcebo  PPaiva  DFaulkner  MSwanson  J Readability of patient education materials. Appl Nurs Res. 1996;9139- 143Article
Williams  DMCounselman  RLCaggiano  CD Emergency department discharge instructions and patient literacy. Am J Emerg Med. 1996;1419- 22Article
Dollahite  JThompson  CMcNew  R Readability of printed sources of diet and health information. Patient Educ Couns. 1996;27123- 134Article
Bauman  A The comprehensibility of asthma education materials. Patient Educ Couns. 1997;32 ((1 suppl)) S51- S59Article
Dowe  MCLawrence  PACarlson  JKeyserling  TC Patients' use of health-teaching materials at three readability levels. Appl Nurs Res. 1997;1086- 93Article
Mumford  ME A descriptive study of the readability of patient information leaflets designed by nurses. J Adv Nurs. 1997;26985- 991Article
Wilson  FL The suitability of United States Pharmacopoeia dispensory information psychotropic drug leaflets for urban patients with limited reading skills. Arch Psychiatr Nurs. 1999;13204- 211Article
Busselman  KMHolcomb  CA Teaching skills and comprehension of the dietary guidelines by WIC participants. J Am Diet Assoc. 1994;94622- 625Article
Davis  TCMayeaux  EJFredrickson  DBocchini  JAJackson  RHMurphy  PW Reading ability of parents compared with reading level of patient education materials. Pediatrics. 1994;93460- 468
Fredrickson  DDWashington  RLPham  NJackson  TWiltshire  JJecha  LD Reading grade levels and health behaviors of parents at child clinics. Kans Med. 1995;96127- 129
Klingbell  CSpeece  MWSchubiner  H Readability of pediatric patient education materials. Clin Pediatr (Phila). 1995;3496- 102Article
Graber  MARoller  CMKaeble  B Readability levels of patient education material on the World Wide Web. J Fam Pract. 1999;4858- 61
Tang  PCJaworski  MAFellencer  CA  et al.  Methods for assessment information needs of clinicans in ambulatory care. Gardner  Red.Proceedings of the 19th Annual ymposium on Computer Applications in Medical Care Philadelphia, Pa Hanley & Belfus 1995630- 634
Murray  P Click here—and be disappointed? evaluating websites. Comput Nurs. 1996;14260- 261
Impicciatore  PPandolfini  CCasella  NBonati  M Reliability of health information for the public on the World Wide Web: systematic survey of advice on managing fever in children at home. BMJ. 1997;3141875- 1879Article
Silberg  WMLundberg  GDMusacchio  RA Assessing, controlling, and assuring the quality of medical information on the Internet: caveat lector et viewor—let the reader and viewer beware. JAMA. 1997;2771244- 1245Article
Wyatt  JC Commentary: measuring quality and impact of the World Wide Web. BMJ. 1997;3141879- 1881Article
Berry  EParker-Jones  CJones  RG  et al.  Systematic assessment of World Wide Web materials for medical education: online, cooperative peer review. J Am Med Inform Assoc. 1998;5382- 389Article
Jadad  ARGagliardi  A Rating health information on the Internet: navigating to knowledge or to Babel? JAMA. 1998;279611- 614Article
Sinha  MKennedy  C Evaluation of parent-oriented health information for childhood asthma on the Internet: more quantity than quality!  Presented at: the 38th Annual Meeting of the Ambulatory Pediatric Association May 1-5, 1998 New Orleans, La
Fry  E Readability formula that saves time. J Reading. 1986;7512- 516575- 578
McLaughlin  GH SMOG grading: a new readability formula. J Reading. 1969;12639- 646
Flesch  R A new readability yardstick. J Appl Psychol. 1948;32221- 33Article
Kincaid  JPFishburne  RPRobers  RLChissom  BS Derivation of New Readabiity Formulas (Automated Reliability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel.  Memphis, Tenn Naval Air Station1975;Research Branch Report 8-75
Mailloux  SLJohnson  MEFisher  DGPettibone  TJ How reliable is computerized assessment of readability? Comput Nurs. 1995;13221- 225
Gaston  NDaniels  P Guidelines: writing for adults with limited reading skills. Available at September 18, 2000
Smith  S Readability testing health information. Available at September 18, 2000
Frederickson  D Polio vaccine information pamphlets: study of parent comprehension comparing a short polio vaccine information pamphlet containing graphics and simple language with the currently available Public Health Service brochure.  Presented at: the annual conference of the Ambulatory Pediatric Association 1993 Seattle, Wash
Giordano  BP Ensuring the readability of patient education materials is one way to demonstrate perioperative nurses' value. AORN J. 1996;63699- 700Article
Lynch  PJHorton  S Web style guide: basic design principles for creating websites. Available at March 1, 2000
Nielsen  J Designing Web Usability.  Indianapolis, Ind New Riders Publishing2000;
Rabbi  AT Determining difficulty levels of text written in languages other than English. Zakaluk  BLSamuels  SJeds.Readability: Its Past, Present, and Future Newark, Del International Reading Association1988;
Lawrence  SGiles  CL Accessibility of information on the Web. Nature. 1999;400107- 109Article