[Skip to Content]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Download PDF
Table 1. Characteristics of the Study Sample
Table 1. Characteristics of the Study Sample
Table 2. Task Performance and Usability Ratings (n = 112)
Table 2. Task Performance and Usability Ratings (n = 112)
1.
 Expanding the reach and impact of consumer e-Health tools. US Department of Health and Human Services. http://www.health.gov/communication/ehealth/ehealthTools/executivesummary.htm. Accessed July 24, 2008
2.
Folstein MF, Folstein SE, McHugh PR. Mini-mental state: a practical method for grading the cognitive state of patients for the clinician.  J Psychiatr Res. 1975;12(3):189-198PubMedArticle
3.
Czaja SJ, Charness N, Dijkstra K, Fisk AD, Rogers WA, Sharit J. Computer and technology experience questionnaire [CREATE Technical Rep No. CREATE-2006-03]. 2006
4.
Czaja SJ, Charness N, Dijkstra K, Fisk AD, Rogers WA, Sharit J. Demographic and background questionnaire [CREATE Technical Rep No. CREATE-2006-02]. 2006
5.
Baker DW, Williams MV, Parker RM, Gazmararian JA, Nurss J. Development of a brief test to measure functional health literacy.  Patient Educ Couns. 1999;38(1):33-42PubMedArticle
6.
 HyperCam [computer program]. Murrysville, PA: Hyperionics Technology; 2006
Citations 0
Research Letter
August 20, 2008

Usability of the Medicare Health Web Site

JAMA. 2008;300(7):790-792. doi:10.1001/jama.300.7.790-b

To the Editor: The use of Internet-based applications to support health management tasks has grown enormously.1 Studies investigating these applications have focused on prevalence of or reasons for use. We evaluated the ability of a sample of older adults to use the Medicare.gov Web site to make decisions concerning eligibility for services and prescription drug plans.

Methods

Two hundred eighteen people responded to newspaper advertisements and flyers in community and senior centers in the greater Miami area through telephone contact. Participants were required to be aged at least 50 years; be cognitively unimpaired (Mini-Mental State Examination2 score >26); have had computer experience (assessed via questionnaire3); be English speaking; and not have depressive symptoms or severe visual or hearing impairments. Following screening, 201 were eligible for participation, 64 of whom expressed lack of interest or did not report for the study. Participants were enrolled consecutively; 112 completed the protocol and were compensated $50 (Table 1). To characterize the sample, data were collected on age, education, and race/ethnicity (based on self-report using fixed categories4). Data were collected at the University of Miami between February 2006 and May 2007. The study was approved by the university's institutional review board, and all participants provided written consent.

Participants were asked to use the Medicare.gov Web site to (1) determine eligibility for home health care services, (2) select a home health agency to meet specified needs, (3) make a decision about enrollment in Medicare Part D based on specified criteria, and (4) select a drug plan and determine associated costs based on a specified medication regime. They provided written responses. Participants also evaluated Web site usability and the value of Internet health information via questionnaire. Responses were rated by 2 investigators using a scoring sheet and through analysis of video records.6 Interrater reliability on 2 problems in a random sample of 20 participants was good (r = 0.96 and r = 0.97).

During data collection, changes were made to the appearance of the home page and sizing of selection buttons on the Web site; content was not altered. To assess comparability, we compared performance scores, usability ratings, and characteristics of participants who used the initial (n = 82) and modified (n = 30) versions using χ2 tests, t tests, and Wilcoxon tests. No differences were found for ratings of usability or performance scores. A greater percentage of Hispanic (46.2%) and non-Hispanic white (44.9%) participants used the initial version than non-Hispanic black participants (9.0%), but more non-Hispanic black participants (42.9%) used the modified version than Hispanic (28.6%) or non-Hispanic white participants (28.6%) (P < .001). Analyses were performed using SPSS 15.0 (SPSS Inc, Chicago, Illinois), with 2-sided significance set at P < .05.

Results

Most participants were unable to specify all eligibility criteria for home health services (n = 77 [68.8%; 95% confidence interval {CI}, 59.8%-76.8%]), choose the correct home health agency (n = 90 [80.4%; 95% CI, 72.3%-86.9%]), or execute computation procedures needed for making a plan enrollment decision (n = 94 [83.9%; 95% CI, 76.3%-89.8%]). Only about half (n = 64 [57.1%; 95% CI, 47.9%-66.0%]) were able to make an enrollment decision. Most participants (n = 81 [72.3%; 95% CI, 63.5%-80.0%]) were unable to select a drug plan or had problems navigating to the necessary Web pages, locating needed information, or executing actions required for plan selection. Participants also reported problems with usability (Table 2), but almost all indicated that the Internet was a valuable source of health information.

Comment

A sample of relatively well-educated adults with computer experience had difficulty using the Medicare.gov Web site to determine eligibility for services and enroll in a drug plan. Participants also reported problems navigating within the Web site and locating information. Although the sample was relatively small and restricted to people aged 50 years or more, it was diverse and larger than samples included in most usability studies. It is likely that persons with less computer experience would have even greater difficulty. To ensure that electronic health tools reach their full potential, broad and inclusive input from consumers should serve as the basis for design.

Back to top
Article Information

Author Contributions: Mr Nair had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Czaja, Sharit, Nair.

Acquisition of data: Czaja.

Analysis and interpretation of data: Czaja, Sharit, Nair.

Drafting of the manuscript: Czaja, Nair.

Critical revision of the manuscript for important intellectual content: Czaja, Sharit.

Statistical analysis: Czaja, Nair.

Obtained funding: Czaja, Sharit.

Administrative, technical, or material support: Nair.

Study supervision: Czaja, Sharit.

Financial Disclosures: None reported.

Funding Support: This research was supported by grant 2 P01 AG017211 from the National Institutes of Health/National Institute on Aging.

Role of the Sponsor: The National Institutes of Health played no role in the design and conduct of the study; the collection, management, analysis, and interpretation of the data; or the preparation, review, or approval of the manuscript.

Additional Contributions: Tamar El-Attar, MS, University of Miami Department of Industrial Engineering, and Mario Hernandez, MS, Center on Aging, University of Miami, both provided assistance in data collection. Chin Chin Lee, MS, MSPH, Center on Aging, University of Miami, provided administrative assistance and study supervision. Shih Hua Fu, MS, Center on Aging, University of Miami, provided technical assistance. No one received compensation for their roles.

References
1.
 Expanding the reach and impact of consumer e-Health tools. US Department of Health and Human Services. http://www.health.gov/communication/ehealth/ehealthTools/executivesummary.htm. Accessed July 24, 2008
2.
Folstein MF, Folstein SE, McHugh PR. Mini-mental state: a practical method for grading the cognitive state of patients for the clinician.  J Psychiatr Res. 1975;12(3):189-198PubMedArticle
3.
Czaja SJ, Charness N, Dijkstra K, Fisk AD, Rogers WA, Sharit J. Computer and technology experience questionnaire [CREATE Technical Rep No. CREATE-2006-03]. 2006
4.
Czaja SJ, Charness N, Dijkstra K, Fisk AD, Rogers WA, Sharit J. Demographic and background questionnaire [CREATE Technical Rep No. CREATE-2006-02]. 2006
5.
Baker DW, Williams MV, Parker RM, Gazmararian JA, Nurss J. Development of a brief test to measure functional health literacy.  Patient Educ Couns. 1999;38(1):33-42PubMedArticle
6.
 HyperCam [computer program]. Murrysville, PA: Hyperionics Technology; 2006
×