• This article reviews some methodological aspects of studies of diagnostic reliability in psychiatry. We define and discuss the concept of interrater reliability and review some of the ways in which the design of the reliability study can influence the results. Three basic methodological issues are raised, including: the importance of structured interviews and objective diagnostic criteria, the importance of a test/retest vs an interviewer/ observer design, and the calculation of reliability in a way that takes chance agreement into account.
Helzer JE, Robins LN, Taibleson M, Woodruff RA, Reich T, Wish ED. Reliability of Psychiatric DiagnosisI. A Methodological Review. Arch Gen Psychiatry. 1977;34(2):129-133. doi:10.1001/archpsyc.1977.01770140019001