On Agreement Indices for Nominal Data

Abstract

In empirical research certain measurements are frequently performed. Measurement is the assigning of numbers to subjects to denote a property. These measurements take place at a level of measurement: nominal, ordinal, interval or higher. Especially in the social sciences, the assignment to a category is very often based on the judgement of an expert and not on an objective and clear criterium. Still it is desirable that the 'scientists agree', i.e. the assignment will not or scarcely differ if another rater had performed the assignment task. So it is of importance that there is agreement among the raters. There is agreement when raters assign an entity to the same category. Agreement is not only investigated within the social sciences, but in general also in the fields of biology and medicine.