Kappa coefficient: a popular measure of rater agreement

Summary In mental health and psychosocial studies it is often necessary to report on the between-rater agreement of measures used in the study. This paper discusses the concept of agreement, highlighting its fundamental difference from correlation. Several examples demonstrate how to compute the kappa coefficient – a popular statistic for measuring agreement – both by hand and by using statistical software packages such as SAS and SPSS. Real study data are used to illustrate how to use and interpret this coefficient in clinical research and practice. The article concludes with a discussion of the limitations of the coefficient.

• Publication year

  2015

• Venue

  Shanghai Archives of Psychiatry

• Publication date

  2015-02-25

• Fields of study

  Medicine, Computer Science, Psychology

• Identifiers
  DOI 10.11919/j.issn.1002-0829.215010PMID 25852260PMCID 4372765
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Applied Categorical and Count Data Analysis

  2015cited by this paper
• Applied Categorical and Count Data Analysis

  2012cited by this paper
• Detection of depression in older adults by family and friends: distinguishing mood disorder signals from the noise of personality and everyday life

  2010cited by this paper
• Web-based Supplementary Materials for “ Inference for Kappas for Longitudinal Study Data : Applications to Sexual Health Research ”

  2007cited by this paper
• High agreement but low kappa: I. The problems of two paradoxes.

  1990cited by this paper
• A concordance correlation coefficient to evaluate reproducibility.

  1989cited by this paper
• Intraclass correlations: uses in assessing rater reliability.

  1979cited by this paper
• The measurement of observer agreement for categorical data.

  1977influential reference
• A Coefficient of Agreement for Nominal Scales

  1960cited by this paper

• How to present the regional agricultural elements from the perspective of landscape in urban context? a case study of Huzhou east station square in China

  2026cites this paper
• Threshold-driven modeling of urban heat exposure under impervious surface expansion: a decadal remote sensing assessment of Shanghai

  2026cites this paper
• Settlements Extraction and Spatiotemporal Analysis with SAM and Random Forest from High-Resolution Remote Sensing

  2026cites this paper
• NLP-Based Review for Toxic Comment Detection Tailored to the Chinese Cyberspace

  2026cites this paper
• Intelligent tool wear monitoring approach in milling of titanium alloys

  2026cites this paper
• Effects of different non-pharmacological interventions on executive function in healthy older people: A systematic review and network meta-analysis.

  2026cites this paper
• Home medication review and drug-related problems in patients with chronic diseases at primary health centers in Yogyakarta, Indonesia: A cross-sectional multicenter study

  2026cites this paper
• Modeling the spread of forest fires through cellular automata by leveraging deep learning to derive transition rules

  2025cites this paper
• Integrating multiple terrain features for artefact detection in the newly released TanDEM-X 30 m DEM and DCM over the Loess Plateau

  2025cites this paper
• Viable-sustainable supplier selection and order allocation problem considering Industry 5.0 pillars under mixed uncertainty

  2025cites this paper
• CIKT: A Collaborative and Iterative Knowledge Tracing Framework with Large Language Models

  2025cites this paper
• Impact of mouthguards on the prevention of dentofacial injuries and sports performance among athletes: An umbrella review.

  2025cites this paper
• Evaluation of the synergistic change in cultivated land and wetland in northeast China from 1990 to 2035

  2025cites this paper
• Intra- and inter-rater reliability of anterior and posterior drawer tests for the assessment of people with shoulder instability

  2025cites this paper
• Spatio-Temporal Evolution of Land Use and Carbon Stock Under Multiple Scenarios Based on the PLUS-InVEST Model: A Case Study of Chengdu

  2025cites this paper
• What goals does China’s medical insurance fund supervision aim to achieve? A quantitative analysis based on national-level policies

  2025influential citation
• Spatial Synergy Between Carbon Storage and Emissions in Coastal China: Insights from PLUS-InVEST and OPGD Models

  2025cites this paper
• Using Large Language Models to Assess the Consistency of Randomized Controlled Trials on AI Interventions With CONSORT-AI: Cross-Sectional Survey

  2025cites this paper
• Spatio-temporal patterns of land use and land cover, and their impact on groundwater quality in the industrialized Muvattupuzha basin

  2025cites this paper
• Change Patterns of Ecological Vulnerability and Its Dominant Factors in Mongolia During 2000–2022

  2025cites this paper
• Constraint Satisfaction Problems: better apprehending them through Tangible Interaction, a Systematic Literature Review

  2025cites this paper
• Low agreement and frequent invalid controls in two SARS-CoV-2 T-cell assays in people with compromised immune function

  2025cites this paper
• Geochemical modeling, evaluation of contaminants and impacts of land use, land cover patterns on groundwater quality in Khushab, Pakistan.

  2025cites this paper
• A comprehensive evaluation of a novel targeted-sequencing workflow for Mycobacterium species identification and anti-tuberculosis drug-resistance detection

  2025influential citation
• Acceptance sampling with multiple inspectors: Critical aspects in aggregating individual assessments

  2025cites this paper
• Machine and deep learning for the prediction of nutrient deficiency in wheat leaf images

  2025cites this paper
• Enhanced Neurovascular Imaging Using Ultra-High-Resolution CT and Deep Learning–Based Image Reconstruction

  2025cites this paper
• Landslide spatial prediction using data-driven based statistical and hybrid computational intelligence algorithms

  2025cites this paper
• EEG-ConvoBLSTM: A novel hybrid model for efficient EEG signal classification.

  2025cites this paper
• Power Transformer Fault Diagnosis Method Based on SMOTE and Convolution Soft Threshold Network

  2025cites this paper
• Methodological establishment and diagnostic value of a multiplex fluorescent PCR assay for the detection of three fastidious respiratory pathogens

  2025influential citation
• Validation of an Administrative Claims-based Line of Therapy Algorithm for Women with Ovarian Cancer Using Medical Chart Review

  2025influential citation
• Innovative screening for lower extremity atherosclerotic disease in people with diabetes: using novel and multidimensional PPG features

  2025cites this paper
• Quality of Patient‐Orientated Internet Information on Watch and Wait for Rectal Cancer

  2025cites this paper
• Objective Emotion Assessment Using a Triple Attention Network for an EEG-Based Brain–Computer Interface

  2025cites this paper
• Conceptual construction and scale development of leadership taking charge behavior in the Chinese cultural context

  2025influential citation
• Feasibility and Reliability of Ammer–Coelho Computational Tool for Sex Estimation: A Pilot Study on an Elderly Scottish Sample

  2025influential citation
• A review of deep learning for fundus image enhancement

  2025cites this paper
• Inter- and intra-rater reliability of two aquatic safety skill assessment tools

  2025cites this paper
• Diagnostic performance of the clear cell likelihood score integrated with cystic degeneration or necrosis on MR imaging for identifying clear cell renal cell carcinoma in cT1 solid renal masses

  2025influential citation
• Phonological processing skills influence text-reading fluency in Russian-speaking adolescents

  2025cites this paper
• Bacterial pathogens causing skin and soft tissue infections and antibiotic susceptibility in South Asia: a scoping review protocol

  2025influential citation
• Spatial Simulation of Future Changes in the LULC for Babil Governorate Using Cellular Automata (CA)-Markov Model

  2025cites this paper
• Statistics Primer for Radiologists: Part 2—Advanced statistics for Enhancing Diagnostic Precision and Research Validity

  2025cites this paper
• Integrating Grain–Carbon Synergy and Ecological Risk Assessment for Sustainable Land Use in Mountainous High-Risk Areas

  2025cites this paper
• ANALISIS KETELITIAN KLASIFIKASI PENUTUPAN LAHAN MENGGUNAKAN METODE DIGITIZE ON SCREEN DAN DEEP LEARNING SERIES CONVOLUTIONAL NEURAL NETWORK (CNN) BERDASARKAN CITRA LANDSAT-8 OLI (STUDI KASUS: PROVINSI KALIMANTAN TIMUR)

  2025cites this paper
• Band Selection Algorithm Based on Multi-Feature and Affinity Propagation Clustering

  2025cites this paper
• Out of Distribution, Out of Luck: How Well Can LLMs Trained on Vulnerability Datasets Detect Top 25 CWE Weaknesses?

  2025cites this paper
• Individual Tree Mortality Prediction of Pinus yunnanensis Franch.—Based on Stacking Ensemble Learning and Threshold Optimization

  2025cites this paper
• Using Large Language Models to Assess Tutors' Performance in Reacting to Students Making Math Errors

  2024cites this paper
• Impacts of farming and herding activities on land use and land cover changes in the north eastern corridor of Ghana: A comprehensive analysis

  2024cites this paper
• Concordance and Acceptability of HPV DNA Genotyping Test by Patient’s Self-Sampling Against Clinician Sampling

  2024cites this paper
• Neurosurgical Care of Brain Tumors in Low- and Middle-Income Countries: A Scoping Review of the Workforce Challenges.

  2024cites this paper
• Pilgrim Tourism: Tourist Experience in Accommodation Using Content Analysis

  2024cites this paper
• Delineation of Urban Land Cover Changes Using Remote Sensing in Ninh Kieu District, Can Tho Province, Vietnam

  2024cites this paper
• Analysis of the Involvement of Susceptibility Genes to Coronary Heart Disease in Implementation Signaling and Metabolic Pathways

  2024influential citation
• Caregiver Report of Children’s Exposure to Adverse Life Events: Concordance Between Questionnaire and Interview Approaches

  2024cites this paper
• Rapid Screening of New Psychoactive Substances Using pDART-QqQ-MS

  2024cites this paper
• Construction and validation of machine learning algorithm for predicting depression among home-quarantined individuals during the large-scale COVID-19 outbreak: based on Adaboost model

  2024influential citation
• A landscape index for indicating the spatio-temporal dynamics of carbon storage in an opencast coal mine

  2024cites this paper
• Spatially‐nested hierarchical species distribution models to overcome niche truncation in national‐scale studies

  2024cites this paper
• Geo-spatial analysis of urbanization and environmental changes with deep neural networks: Insights from a three-decade study in Kerch peninsula

  2024cites this paper
• Healthcare setting staff satisfaction in Ethiopia: Systematic review and meta‐analysis

  2024influential citation
• Quantitative Foveal Structural Metrics as Predictors of Visual Acuity in Human Albinism

  2024cites this paper
• Comparing Video Analysis to Computerized Detection of Limb Position for the Diagnosis of Movement Control during Back Squat Exercise with Overload

  2024cites this paper
• Short-term economic evaluation of physical activity-based corporate health programs: a systematic review

  2024influential citation
• Fragility in cardiovascular randomized controlled trials with primary continuous outcomes (2018–2022) from multi-perspectives assessment: a cross-sectional survey

  2024cites this paper
• Validation of the Black-&-White sign to predict intracerebral hematoma expansion in the multi-center PREDICT study cohort

  2024cites this paper
• Crop Classification from Drone Imagery Based on Lightweight Semantic Segmentation Methods

  2024cites this paper
• Use of multivariant enzyme-linked immunosorbent assay (ELISA) in the diagnosis of autoimmune bullous disorders in a resource-limited setting: A single-center experience.

  2024cites this paper
• Behind the Counter: Exploring Motivations and Barriers of Online Counterspeech Writing

  2024cites this paper
• Impact of deep Learning-enhanced contrast on diagnostic accuracy in stroke CT angiography.

  2024cites this paper
• Understanding Non-Verbal Irony Markers: Machine Learning Insights Versus Human Judgment

  2024cites this paper
• The Relationship Between Dyadic Coping and Mental Health in Stroke Survivors and Their Spouse Caregivers: An Actor-Partner Interdependence Model.

  2024cites this paper
• The eyes don’t have it: Coaches’ eye is not a valid method of estimating biological maturation in male Gaelic football

  2024cites this paper
• Leveraging Large Vision-Language Model for Better Automatic Web GUI Testing

  2024influential citation
• The effect of dexmedetomidine-ketamine combination versus dexmedetomidine on behavior of uncooperative pediatric dental patients: a randomized controlled clinical trial

  2024cites this paper
• Urban green space cover change analysis using GIS and remote sensing in two rapidly urbanized cities, Debre Berhan and Debre Markos, Ethiopia

  2024cites this paper
• Spatial–temporal evolution analysis of multi-scenario land use and carbon storage based on PLUS-InVEST model: A case study in Dalian, China

  2024cites this paper
• A novel formula for estimating the vertical dimension of occlusion using panoramic radiography

  2024influential citation
• Exploring barriers and facilitators, and their effectiveness in eye health promotion interventions: Protocol of a systematic review

  2024cites this paper
• Rice Phenology Classification Model Based on Sentinel-1 Using Machine Learning Method on Google Earth Engine

  2024cites this paper
• Effectiveness of buccal administration of dexmedetomidine and ketamine combination in paediatric dental sedation: A randomized controlled clinical trial.

  2024cites this paper
• Enhancing Place Emotion Analysis with Multi-View Emotion Recognition from Geo-Tagged Photos: A Global Tourist Attraction Perspective

  2024cites this paper
• Psychometric adequacy of the persian adapted version of the tilburg frailty indicator (P-TFI)

  2024cites this paper
• EEG emotion recognition based on differential entropy feature matrix through 2D-CNN-LSTM network

  2024cites this paper
• Evaluating landslide susceptibility: the impact of resolution and hybrid integration approaches

  2024cites this paper
• Perceiving and Countering Hate: The Role of Identity in Online Responses

  2024influential citation
• Tumor subtypes and signature model construction based on chromatin regulators for better prediction of prognosis in uveal melanoma

  2023cites this paper
• Non-Intrusive Load Identification Method Based on KPCA-IGWO-RF

  2023cites this paper
• Spontaneous visual database for detecting learning-centered emotions during online learning

  2023cites this paper
• GEE-Based monitoring method of key management nodes in cotton production

  2023cites this paper
• Teachers’ Professional Training through Augmented Reality: A Literature Review

  2023influential citation
• The Study of Risk Factors and The Predict Model of Cerebral Microbleeds.

  2023cites this paper
• Capturing product/service improvement ideas from social media based on lead user theory

  2023cites this paper
• La información y alfabetización informacional del marco común de competencia digital docente: una revisión sistemática

  2023influential citation
• Boosting Distress Support Dialogue Responses with Motivational Interviewing Strategy

  2023influential citation
• Strain Identification and Drug Resistance Analysis of Matrix-Assisted Laser Desorption/Ionization Time-of-Flight Mass Spectrometry in Nontuberculous Mycobacterial Lung Disease

  2023cites this paper
• Ultrafast Brain MRI Protocol at 1.5 T Using Deep Learning and Multi-shot EPI.

  2023cites this paper
• SCS 5th Annual Conference. São Paulo, Brazil, 2022

  2023cites this paper