Estimating kappa from binocular data and comparing marginal probabilities

An alternative procedure is proposed that utilizes existing formulae for the computation of (weighted) kappa and its standard error and the McNemar test is generalized to compare the probabilities of a positive judgement by the two graders.

Abstract

Suppose that two graders classify all eyes in a sample of patients for the presence or absence of a specified abnormality. In the statistical analysis of the data, possible correlation between the observations in the right and left eyes should be taken into account. Recently, general methods have been developed to analyse grader agreement in rating paired body structures. In the present paper, an alternative procedure is proposed that utilizes existing formulae for the computation of (weighted) kappa and its standard error. In addition, the McNemar test is generalized to compare the probabilities of a positive judgement by the two graders.