[Skip to Navigation]
December 1978

Reliability and Validity in Binary Ratings: Areas of Common Misunderstanding in Diagnosis and Symptom Ratings

Author Affiliations

From the Behavior Genetics Center, Department of Psychology, University of Minnesota, Minneapolis. Part of this research was done while the first author was at the Institute of Psychiatry, Denmark Hill, London.

Arch Gen Psychiatry. 1978;35(12):1454-1459. doi:10.1001/archpsyc.1978.01770360058007

• Confusion may exist between the reliability of a binary rating (for example, schizophrenia versus not-schizophrenia) and its implications for validity. High reliability does not guarantee validity, but paradoxically, low reliability does not imply poor validity in all contexts. Changes in the base rate or in experimental design may indicate high validity even when the reliability was thought to be low. Attempts to improve the psychiatric nomanclature by increasing only reliability run the risk of the "attenuation paradox" where further increases in reliability will make the ratings less valid. Finally, the assumption of random error in making diagnoses does not always hold, so that statistical analyses must be adjusted accordingly. New statistical methods are needed to index only false-positive or false-negative rates in order to quantify the error that will reduce some validity coefficients.

Add or change institution