Online Cluster Validity Indices for Streaming Data

Cluster analysis is used to explore structure in unlabeled data sets in a wide range of applications. An important part of cluster analysis is validating the quality of computationally obtained clusters. A large number of different internal indices have been developed for validation in the offline setting. However, this concept has not been extended to the online setting. A key challenge is to find an efficient incremental formulation of an index that can capture both cohesion and separation of the clusters over potentially infinite data streams. In this paper, we develop two online versions (with and without forgetting factors) of the Xie-Beni and Davies-Bouldin internal validity indices, and analyze their characteristics, using two streaming clustering algorithms (sk-means and online ellipsoidal clustering), and illustrate their use in monitoring evolving clusters in streaming data. We also show that incremental cluster validity indices are capable of sending a distress signal to online monitors when evolving clusters go awry. Our numerical examples indicate that the incremental Xie-Beni index with forgetting factor is superior to the other three indices tested.

• Publication year

  2018

• Venue

  arXiv.org

• Publication date

  2018-01-08

• Fields of study

  Mathematics, Computer Science

• Identifiers
  arXiv 1801.02937
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Online Clustering of Multivariate Time-series

  2016cited by this paper
• Reservoir computing compensates slow response of chemosensor arrays exposed to fast varying gas concentrations in continuous monitoring

  2015cited by this paper
• A statistical approach for clustering in streaming data

  2014cited by this paper
• Incremental Clustering: The Case for Extra Clusters

  2014cited by this paper
• A Relevance Weighted Ensemble Model for Anomaly Detection in Switching Data Streams

  2014cited by this paper
• Data stream clustering: A survey

  2013cited by this paper
• An extensive comparative study of cluster validity indices

  2013influential reference
• The ClusTree: indexing micro-clusters for anytime stream mining

  2011cited by this paper
• Evolving Takagi‐Sugeno Fuzzy Systems from Streaming Data (eTS+)

  2010cited by this paper
• Streaming k-means approximation

  2009cited by this paper
• Evolving Fuzzy-Rule-Based Classifiers From Data Streams

  2008cited by this paper
• On Clustering Massive Data Streams: A Summarization Paradigm

  2007cited by this paper
• Density-Based Clustering over an Evolving Data Stream with Noise

  2006cited by this paper
• Clustering Data Streams: Theory and Practice

  2003cited by this paper
• Evaluating the Fit of Structural Equation Models: Tests of Significance and Descriptive Goodness-of-Fit Measures.

  2003cited by this paper
• Sequential Competitive Learning and the Fuzzy c-Means Clustering Algorithms

  1996cited by this paper
• On cluster validity for the fuzzy c-means model

  1995cited by this paper
• An examination of procedures for determining the number of clusters in a data set

  1994cited by this paper
• Segmentation of thermal images using the fuzzy C-means algorithm

  1993cited by this paper
• A Validity Measure for Fuzzy Clustering

  1991cited by this paper
• Pattern Recognition with Fuzzy Objective Function Algorithms

  1981cited by this paper
• A Cluster Separation Measure

  1979influential reference
• Some methods for classification and analysis of multivariate observations

  1967influential reference

• System-forming aspects of computer science and mathematics teachers' readiness to develop and use computer didactic games: a comprehensive TPACK-GPCK framework analysis

  2025cites this paper
• Incremental Cluster Validity Index for Predicting Early Signs of Change in Data Streams

  2023cites this paper
• Incremental Cluster Validity Index-Guided Online Learning for Performance and Robustness to Presentation Order

  2021influential citation
• Evaluating Evolving Structure in Streaming Data With Modified Dunn's Indices

  2021influential citation
• Incremental Cluster Validity Indices for Online Learning of Hard Partitions: Extensions and Comparative Study

  2020influential citation
• The Use of Digital Technologies for Flipped Learning Implementation

  2020cites this paper
• Incremental Cluster Validity Indices for Hard Partitions: Extensions and Comparative Study

  2019influential citation
• Microarray Filtering-Based Fuzzy C-Means Clustering and Classification in Genomic Signal Processing

  2019cites this paper
• Event news detection and citizens community structure for disaster management in social networks

  2019cites this paper
• An empirical comparison of machine learning clustering methods in the study of Internet addiction among students majoring in Computer Sciences

  2019cites this paper
• A New Incremental Cluster Validity Index for Streaming Clustering Analysis

  2019influential citation
• Analysis of Streaming Clustering Using an Incremental Validity Index

  2018cites this paper