Detecting Concept Drift In Medical Triage

in their accompanying referral documents, which contain a mix of free text and structured data. By training a model to predict triage decisions from these referral documents, we can partially automate the triage process, resulting in more efficient and systematic triage decisions. One of the difficulties of this task is maintaining robustness against changes in triage priorities due to changes in policy, funding, staff, or other factors. This is reflected as changes in relationship between document features and triage labels, also known as concept drift. These changes must be detected so that the model can be retrained to reflect the new environment. We introduce a new concept drift detection algorithm for this domain called calibrated drift detection method (CDDM). We evaluated CDDM on benchmark and synthetic medical triage datasets, and find it competitive with state-of-the-art detectors, while also being less prone to false positives from feature drift.

• Publication year

  2020

• Venue

  Annual International ACM SIGIR Conference on Research and Development in Information Retrieval

• Publication date

  2020-07-25

• Fields of study

  Medicine, Computer Science

• Identifiers
  DOI 10.1145/3397271.3401228
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• A large-scale comparison of concept drift detectors

  2018influential reference
• A Reservoir of Adaptive Algorithms for Online Learning from Evolving Data Streams

  2018cited by this paper
• Beyond sigmoids: How to obtain well-calibrated probabilities from binary classifiers with beta calibration

  2017cited by this paper
• RDDM: Reactive drift detection method

  2017cited by this paper
• MIMIC-III, a freely accessible critical care database

  2016cited by this paper
• Online and Non-Parametric Drift Detection Methods Based on Hoeffding’s Bounds

  2015cited by this paper
• Predicting good probabilities with supervised learning

  2005influential reference

• ImageCLEF-Medical 2025: MedCLIP Model for Medical Caption Prediction and Concept Detection

  2025cites this paper
• Keeping Medical AI Healthy and Trustworthy: A Review of Detection and Correction Methods for System Degradation.

  2025cites this paper
• Improving musculoskeletal care with AI enhanced triage through data driven screening of referral letters

  2025cites this paper
• Self-Healing Machine Learning: A Framework for Autonomous Adaptation in Real-World Environments

  2024cites this paper
• Early Concept Drift Detection via Prediction Uncertainty

  2024cites this paper
• MLHOps: Machine Learning for Healthcare Operations

  2023cites this paper
• A survey on detecting healthcare concept drift in AI/ML models from a finance perspective

  2023cites this paper
• A Concept Drift Detection Method for Electricity Forecasting Based on Adaptive Window and Transformer

  2023cites this paper
• RETRACTED: Triaging Medical Referrals Based on Clinical Prioritisation Criteria Using Machine Learning Techniques

  2022cites this paper
• Automated Triaging Medical Referral for Otorhinolaryngology Using Data Mining and Machine Learning Techniques

  2022influential citation
• To Err is (only) Human. Reflections on How to Move from Accuracy to Trust for Medical AI

  2021cites this paper
• The importance of being external. methodological insights for the external validation of machine learning models in medicine

  2021cites this paper
• Beyond Adaptation: Understanding Distributional Changes (Dagstuhl Seminar 20372)

  2020cites this paper