Issues in the Use of Kappa to Estimate Reliability
Medical CarePublished 1 August 1986
Karen L. Soeken, Patricia A. Prescott
Citations105
SJR quartileQ1
SJR score1.57
SNIP1.38
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
Questions regarding the application of the kappa statistic in assessing the reliability of measures to classify patients are discussed and suggestions for a more comprehensive approach to the study of the psychometric properties of measures of patient classification are offered.
Abstract
Selected issues regarding the application of the kappa statistic in assessing the reliability of measures to classify patients are discussed. Data are used to illustrate how the methods used to estimate reliability can influence the resulting coefficient. Suggestions for a more comprehensive approach to the study of the psychometric properties of measures of patient classification are then offered.
Keywords
MedicineDecision SciencesMathematics
BiometricsThe Measurement of Observer Agreement for Categorical Data
78,934 Citations1977J. Richard Landis, Gary G. Koch
A general statistical methodology for the analysis of multivariate categorical data arising from observer reliability studies is presented and tests for interobserver bias are presented in terms of first-order marginal homogeneity and measures of interob server agreement are developed as generalized kappa-type statistics.
BiometricsAn Application of Hierarchical Kappa-type Statistics in the Assessment of Majority Agreement among Multiple Observers
3,808 Citations1977J. Richard Landis, Gary G. Koch
A subset of 'observers who demonstrate a high level of interobserver agreement can be identified by using pairwise agreement statistics betweeni each observer and the internal majority standard opinion on each subject.
Educational and Psychological MeasurementCoefficient Kappa: Some Uses, Misuses, and Alternatives
1,544 Citations1981Robert L. Brennan, Dale J. Prediger
The Journal of Nervous and Mental DiseaseON THE METHODS AND THEORY OF RELIABILITY
1,317 Citations1976John J. Bartko, William T. Carpenter
This paper reviews the most frequently used and misused reliability measures appearing in the mental health literature and suggests some suitable reliability measures to be used.
Medical CareThe Abbreviated Injury Scale and Injury Severity Score
617 Citations1985Ellen J. MacKenzie, Sam Shapiro +1 more
Results showed that as a group, physicians and nurses tend to be more reliable in their ratings than either emergency medical technicians (EMTs) or nonclinical technicians, although a research assistant who is well trained in AIS coding and is a diligent worker can use the AIS to code severity as reliably as the physicians when sufficient information is provided in the medical chart.
Psychological BulletinIntegration and generalization of kappas for multiple raters.
513 Citations1980Anthony J. Conger
BiometricsA One-Way Components of Variance Model for Categorical Data
468 Citations1977J. Richard Landis, Gary G. Koch
BiometricsExtension of the Kappa Coefficient
235 Citations1980Helena C. Kraemer
An extension of the kappa coefficient is proposed which is appropriate for use with multiple observations per subject and for multiple response choices per observation and to illustrate new approaches to difficult problems in evaluation of reliability.
Applied Psychological MeasurementTesting the Normal Approximation and Minimal Sample Size Requirements of Weighted Kappa When the Number of Categories is Large
187 Citations1981Domenic V. Cicchetti
Medical CareMeasuring Severity of Illness: Homogeneous Case Mix Groups
146 Citations1983Susan D. Horn, Phoebe D. Sharkey +1 more
It is found that this index, which takes into account patient severity of illness, produces groups that are more homogeneous than those produced by four other case mix grouping methods that do not reflect the total burden of the patient's illness.
American Journal of Public HealthMeasuring Severity of Illness: Comparisons Across Institutions
107 Citations1983Susan D. Horn
Medical CareThe Measurement of Hospital Case Mix
96 Citations1982Wanda W. Young, Robert B. Swinkola +1 more
The design and preliminary results of research being conducted by Blue Cross of Western Pennsylvania to measure hospital case mix are described and the model of patient management used in this research interrelates symptoms, diagnosis and treatment is presented.
Statistics in MedicineAn application of kappa‐type analyses to interobserver variation in classifying chest radiographs for pneumoconiosis
39 Citations1984David C. Musch, J. Richard Landis +3 more
Variation in the classification of profusion of small opacities was greater in films of poor quality, and also increased with the length of time between taking and reading the film (film age), and the effect of film age was not controlled by controlling for film quality.
Medical CareMeasuring Severity of Illness: A Reliability Study
38 Citations1983Susan D. Horn, Bette Chachich +1 more
A new severity-of-illness index is described that is generic and may be applied to almost all hospital patients and the reliability of this index and the time required for different types of raters to apply it are examined.
Medical CareDetermining the Generalizability of Rating Scales in Clinical Settings
29 Citations1981William J. Evans, C. Gene Cayten +1 more
The FLS was found to be capable of differentiating between patients and the effect of time on patient status, regardless of the physician rater(s), illustrating how one multi-faceted design can answer questions that formerly required several separate data sets.
Medical CareThe Dependability of Medical Encounter Diagnostic Information
11 Citations1984Dennis A. Revicki
This investigation illustrates the application of a relatively new approach to practical measurement problems, generalizability theory, by evaluating the dependability of medical billing diagnostic information using a rating scale developed by Studney and Hakstian.
American Journal of Public HealthPaying hospitals: how does a severity measure help?
9 Citations1983C Müller
British Journal of Mathematical and Statistical PsychologyMonotonic measures of agreement for ranked data
4 Citations1984Gordon R. Stavig