Inference About Weighted Kappa in the Non-Null Case

Abstract

The accuracy of the large sample standard error of weighted kappa appropriate to the non-null case was studied by computer simulation. Results indi cate that only moderate sample sizes are required to test the hypothesis that two independently de rived estimates of weighted kappa are equal. How ever, in most instances the minimal sample sizes re quired for setting confidence limits around a single value of weighted kappa are inordinately large. An alternative, but as yet untested procedure for set ting confidence limits, is suggested as being poten tially more accurate.