[Skip to Content]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address 54.197.66.254. Please contact the publisher to request reinstatement.
[Skip to Content Landing]
Article
July 13, 1994

Instruments for Assessing the Quality of Drug Studies Published in the Medical Literature

Author Affiliations

From the Institute for Health Policy Studies, School of Medicine (Drs Cho and Bero), and Division of Clinical Pharmacy, School of Pharmacy (Dr Bero), University of California-San Francisco; Center for Health Care Evaluation, Department of Veterans Affairs, Veterans Affairs Medical Center, Palo Alto, Calif (Dr Cho); and Department of Health Research and Policy, Stanford (Calif) University (Dr Cho).

JAMA. 1994;272(2):101-104. doi:10.1001/jama.1994.03520020027007
Abstract

Objective.  —To develop valid and reliable instruments to assess the methodologic quality and clinical relevance of drug studies.

Design.  —We developed an instrument to assess the methodologic quality of articles reporting clinical research and an instrument to measure nonmethodologic measures of quality, such as clinical relevance, generalizability, and adherence to ethical standards. Each instrument was pretested by seven independent, masked reviewers and modified based on interrater agreement and content validity of individual items. We determined correlational validity of the final methodologic quality instrument by comparing quality scores assigned to 10 articles by means of our instrument and a previously published one.

Participants.  —Clinical drug studies published in symposium proceedings and peer reviewed biomedical literature.

Main Outcome Measures.  —Interrater reliability of overall quality scores, measured by intraclass correlation (r) and Kendall's coefficient of concordance (W), and interrater reliability of individual items, by percentage agreement.

Main Results.  —The interrater reliability of the pretest methodologic quality instrument was high (r=.89 [95% confidence interval,.73 to.96]; W=0.64). Correlational validity of the final instrument was suggested by the high degree of concordance with another previously published one (W=0.74). The interrater reliability of the pretest clinical relevance instrument was moderate (r=.41 [95% confidence interval,.18 to.64]; W=0.47). Reviewers confirmed the content validity of both instruments.

Conclusions.  —The two instruments we developed, one measuring methodologic quality and one measuring clinical relevance of articles reporting clinical research, are reliable, valid, and applicable to a variety of research designs.(JAMA. 1994;272:101-104)

×