[Skip to Navigation]
[Skip to Navigation Landing]
Article
July 1965

Rating the Rater: An Experimental Approach to the Methodological Problem of Interrater Agreement

Author Affiliations

VERDUN, QUEBEC
From the Verdun Protestant Hospital.

Arch Gen Psychiatry. 1965;13(1):67-75. doi:10.1001/archpsyc.1965.01730010069009
Abstract

Introduction  IN psychiatric clinical research, the description and evaluation of behavior is very often based on judgements made by human observers. Of the various psychometric methods employing human judgement, the rating scale is perhaps the most widely used because it is easy to administer and usually yields quantifiable data. The apparent simplicity of the method, however, is deceiving, for the rating scale is subject to error and considerable care must be taken to ensure the validity and consistency of the observers' judgements. To establish such consistency of judgements, the usual method is to systematically train all those who are expected to use a certain rating scale. This is done within the setting where the study will be undertaken and prior to its beginning. It results mostly in a high interrater reliability.However, there are certain reasons why it might be desirable to know how closely different

×