Concept Drift Detection by Tracking Weighted Prediction Confidence of Incremental Learning

Data stream mining is great significant in many real-world scenarios, especially in the big data area. However, conventional machine learning algorithms are incapable to process because of its two characteristics (1) potential unlimited number of data is generated in real-time way, it is impossible to store all the data (2) evolving over time, namely, concept drift, will influence the performance of predictor trained on previous data. Concept drift detection method could detect and locate the concept drift in data stream. However, existing methods only utilize the prediction result as indicator. In this article, we propose a weighted concept drift indicator based on incremental ensemble learning to detect the concept. The indicator not only considers the prediction result, but the change of prediction stability of predictor with occurs of concept drift. Also, an incremental ensemble learning based on vote mechanism is especially used to get constantly updated value of indicator. Based on the experiment result on both benchmark and real-world dataset, our method could effectively detect concept drift and outperform other existing methods.

• Publication year

  2022

• Venue

  International Conference on Image, Video and Signal Processing

• Publication date

  2022-03-18

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1145/3531232.3531264
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Preprocessed dynamic classifier ensemble selection for highly imbalanced drifted data streams

  2021cited by this paper
• Dynamical analysis of rumor spreading model in multi-lingual environment and heterogeneous complex networks

  2020cited by this paper
• Learning under Concept Drift: A Review

  2019cited by this paper
• Scikit-Multiflow: A Multi-output Streaming Framework

  2018cited by this paper
• Ensemble learning for data stream analysis: A survey

  2017cited by this paper
• A Systematic Study of Online Class Imbalance Learning With Concept Drift

  2017cited by this paper
• Temporal stability in predictive process monitoring

  2017cited by this paper
• Hierarchical Change-Detection Tests

  2017cited by this paper
• A concept drift-tolerant case-base editing technique

  2016cited by this paper
• Fast Unsupervised Online Drift Detection Using Incremental Kolmogorov-Smirnov Test

  2016cited by this paper
• Online and Non-Parametric Drift Detection Methods Based on Hoeffding’s Bounds

  2015cited by this paper
• A survey on concept drift adaptation

  2014cited by this paper
• Concept Drift Detection and Model Selection with Simulated Recurrence and Ensembles of Statistical Detectors

  2013cited by this paper
• Learning from Time-Changing Data with Adaptive Windowing

  2007cited by this paper
• Early Drift Detection Method

  2005cited by this paper
• Learning with Drift Detection

  2004cited by this paper
• Knowledge-Based Systems

  year unknowncited by this paper

• Using Causal Inference to Solve Uncertainty Issues in Dataset Shift

  2024cites this paper
• A systematic review on detection and adaptation of concept drift in streaming data using machine learning techniques

  2024cites this paper
• Synergistic application of neuro-fuzzy mechanisms in advanced neural networks for real-time stream data flux mitigation

  2024cites this paper
• On the Connection between Concept Drift and Uncertainty in Industrial Artificial Intelligence

  2023cites this paper
• Resilience and Resilient Systems of Artificial Intelligence: Taxonomy, Models and Methods

  2023cites this paper
• Impact of Extreme Class-Imbalance on Landslide-Risk Prediction and Mitigation Using Two-Stage Deep Neural Network

  2022cites this paper