Kappa Reliabilities for Continuous Behaviors and Events

Abstract

Cohen's kappa for measuring agreement between two observers using a discrete nominal scale is extended to measuring agreement over time for continuous nominal scales. The continuous kappa coefficient avoids problems encountered by the arbitrary division of real time durations into presence/absence frequencies in discrete intervals. The extension is simple but issues of independence and number of observations pose problems for significance testing.