Special Article
October 22, 2001

Rigorous New Approach to Constructing a Gold Standard for Validating New Diagnostic Criteria, as Exemplified by the Eosinophilia-Myalgia Syndrome

Author Affiliations

From the Los Alamos Medical Center, Los Alamos, NM (Dr Hertzman); Division of Rheumatology, Immunology, and Allergy, Department of Medicine, Georgetown University School of Medicine, Washington, DC (Dr Clauw); Division of Rheumatology, Department of Internal Medicine, Mayo Clinic, Rochester, Minn (Dr Duffy); Division of Rheumatology and Clinical Immunology, Department of Medicine, University of Pittsburgh School of Medicine, Pittsburgh, Pa (Dr Medsger); and Department of Internal Medicine and Epidemiology, Yale University School of Medicine, New Haven, Conn (Dr Feinstein).


Copyright 2001 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.2001

Arch Intern Med. 2001;161(19):2301-2306. doi:10.1001/archinte.161.19.2301

Background  Constructing diagnostic criteria, a common problem in clinical medicine, is particularly difficult for diseases that lack a pathognomonic "gold standard." To develop an improved strategy for constructing such criteria, we used the eosinophilia-myalgia syndrome as an example. The goal, for research classifications, was to construct validated clinically sensible criteria and to develop improved methods that can be used for other disorders.

Methods  Using a "pattern-based" approach with data from several separate sources, a committee of investigators first prepared and informally tested criteria for the diagnosis of eosinophilia-myalgia syndrome. A gold standard challenge set of reports of cases and noncases was independently generated and separately validated by an external panel of clinical experts. The criteria were then tested using the gold standard set, and interobserver variability and diagnostic accuracy were determined.

Results  Interobserver variability showed the following mean proportionate agreements: 98.7% for the presence of specific criteria elements, 99% to 100% for diagnosis, and 97% to 98% for diagnostic pattern. κ Values were correspondingly high. Diagnostic accuracy showed sensitivity at 88%, specificity at 97%, and overall accuracy at 92%.

Conclusions  The proposed criteria are accurate and reproducible, and can be used in future clinical investigations of the eosinophilia-myalgia syndrome. The new strategy and methods developed for this challenge can be valuable for solving analogous problems in constructing criteria for other clinical disorders.