[Skip to Navigation]
July 1967

Quantification of Agreement in Psychiatric Diagnosis: A New Approach

Author Affiliations

New York
From the Research Division, Washington Heights Community Service, New York State Psychiatric Institute, and Biometrics Research, New York State Department of Mental Hygiene (Dr. Spitzer); New York University (Dr. Cohen); Department of Biostatistics, Columbia University School of Public Health (Mr. Fleiss); and Biometrics Research, New York State Department of Mental Hygiene (Dr. Endicott).

Arch Gen Psychiatry. 1967;17(1):83-87. doi:10.1001/archpsyc.1967.01730250085012

THE LAST few years have seen a number of studies dealing with the problem of the reliability of psychiatric diagnoses. What makes it difficult to assimilate the various findings is the lack of uniform methods for quantifying the salient features of the data. Thus, one study will report an overall rate of perfect agreement of 54%,1 while another will report an overall contingency coefficient of 0.714.2 Still another will report that, given that one diagnostician has made a particular diagnosis, the probability that another diagnostician will make the same diagnosis is 0.57.3

Furthermore, as generally used, all of these methods suffer from one or more deficiencies which are illustrated using the hypothetical data of Table 1. (1) Chance agreement is not taken into account, or equivalently, the base rates at which the various diagnoses are made are not used to qualify the agreement measure.

Add or change institution