Incremental Cluster Validity Indices for Hard Partitions: Extensions and Comparative Study

Validation is one of the most important aspects of clustering, but most approaches have been batch methods. Recently, interest has grown in providing incremental alternatives. This paper extends the incremental cluster validity index (iCVI) family to include incremental versions of Calinski-Harabasz (iCH), I index and Pakhira-Bandyopadhyay-Maulik (iI and iPBM), Silhouette (iSIL), Negentropy Increment (iNI), Representative Cross Information Potential (irCIP) and Representative Cross Entropy (irH), and Conn_Index (iConn_Index). Additionally, the effect of under- and over-partitioning on the behavior of these six iCVIs, the Partition Separation (PS) index, as well as two other recently developed iCVIs (incremental Xie-Beni (iXB) and incremental Davies-Bouldin (iDB)) was examined through a comparative study. Experimental results using fuzzy adaptive resonance theory (ART)-based clustering methods showed that while evidence of most under-partitioning cases could be inferred from the behaviors of all these iCVIs, over-partitioning was found to be a more challenging scenario indicated only by the iConn_Index. The expansion of incremental validity indices provides significant novel opportunities for assessing and interpreting the results of unsupervised learning.

• Publication year

  2019

• Venue

  arXiv.org

• Publication date

  2019-02-18

• Fields of study

  Mathematics, Computer Science

• Identifiers
  arXiv 1902.06711
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Evaluating Evolving Structure in Streaming Data With Modified Dunn's Indices

  2021cited by this paper
• Dual vigilance fuzzy adaptive resonance theory

  2019cited by this paper
• Information State: A Representation for Dynamic Processes Using Information Theory

  2018cited by this paper
• An Information-Theoretic-Cluster Visualization for Self-Organizing Maps

  2018cited by this paper
• A study on exploiting VAT to mitigate ordering effects in Fuzzy ART

  2018cited by this paper
• Distributed dual vigilance fuzzy adaptive resonance theory learns online, retrieves arbitrarily-shaped clusters, and mitigates order dependence

  2018cited by this paper
• Online Cluster Validity Indices for Streaming Data

  2018influential reference
• Analysis of Incremental Cluster Validity for Big Data Applications

  2018cited by this paper
• Online cluster validity indices for performance monitoring of streaming data clustering

  2018influential reference
• Analysis of Streaming Clustering Using an Incremental Validity Index

  2018cited by this paper
• An Analysis of Information Dynamic Behavior Using Autoregressive Models

  2017cited by this paper
• Validity index-based vigilance test in adaptive resonance theory neural networks

  2017cited by this paper
• A Support System for Clustering Data Streams with a Variable Number of Clusters

  2016cited by this paper
• New efficient clustering quality indexes

  2016cited by this paper
• An Approach to Silhouette and Dunn Clustering Indices Applied to Big Data in Spark

  2016cited by this paper
• On Measures of Entropy and Information

  2015cited by this paper
• On comparing partitions

  2015influential reference
• New Quality Indexes for Optimal Clustering Model Identification with High Dimensional Data

  2015cited by this paper
• Advanced ART2 scheme for enhancing metrology-data-quality evaluation

  2014cited by this paper
• NOTE ON FUZZY SETS

  2014cited by this paper
• Information-theoretic clustering: A representative and evolutionary approach

  2013cited by this paper
• Representative cross information potential clustering

  2013cited by this paper
• Signal Processing

  2013cited by this paper
• An extensive comparative study of cluster validity indices

  2013cited by this paper
• A New Validity Index for Fuzzy Clustering

  2012cited by this paper
• A Comparison Study of Validity Indices on Swarm-Intelligence-Based Clustering

  2012cited by this paper
• Center-Wise Intra-Inter Silhouettes

  2012cited by this paper
• BARTMAP: A viable structure for biclustering

  2011cited by this paper
• A Validity Index for Prototype-Based Clustering of Data Sets With Complex Cluster Structures

  2011cited by this paper
• Normality-based validation for crisp clustering

  2010cited by this paper
• Relative clustering validity criteria: A comparative overview

  2010cited by this paper
• Cluster Validation

  2009cited by this paper
• Exploiting Data Topology in Visualization and Clustering of Self-Organizing Maps

  2009influential reference
• Using the Negentropy Increment to Determine the Number of Clusters

  2009cited by this paper
• ART properties of interest in engineering applications

  2009cited by this paper
• CVAP: Validation for Cluster Analyses

  2009cited by this paper
• Autonomous Military Robotics: Risk, Ethics, and Design

  2008cited by this paper
• Extensions of vector quantization for incremental clustering

  2008cited by this paper
• Improved Fuzzy Adaptive Resonance Theory Combining Overlapped Category in Consideration of Connections

  2008cited by this paper
• A new cluster validity index for prototype based clustering algorithms based on inter- and intra-cluster density

  2007cited by this paper
• The Bayesian ARTMAP

  2007cited by this paper
• Evolving clusters in gene-expression data

  2006cited by this paper
• Data topology visualization for the Self-Organizing Map

  2006cited by this paper
• Information Fusion and Situation Awareness using ARTMAP and Partially Observable Markov Decision Processes

  2006cited by this paper
• Survey of clustering algorithms

  2005cited by this paper
• Information Theoretic Learning

  2005cited by this paper
• Evolutionary algorithms for clustering gene-expression data

  2004influential reference
• Validity index for crisp and fuzzy clusters

  2004cited by this paper
• Robust data clustering

  2003cited by this paper
• Cluster validity methods: part I

  2002cited by this paper
• Clustering validity checking methods: part II

  2002cited by this paper
• Information Theoretic Clustering

  2002cited by this paper
• A Maximum Variance Cluster Algorithm

  2002influential reference
• Nonparametric genetic clustering: comparison of validity indices

  2001cited by this paper
• Ellipsoid ART and ARTMAP for incremental clustering and classification

  2001cited by this paper
• Hypersphere ART and ARTMAP for unsupervised and supervised, incremental learning

  2000cited by this paper
• A new clustering evaluation function using Renyi's information potential

  2000cited by this paper
• Clustering of the self-organizing map

  2000cited by this paper
• Chameleon: Hierarchical Clustering Using Dynamic Modeling

  1999cited by this paper
• The use of linked line segments for cluster representation and data reduction

  1999cited by this paper
• Multi-resolution fuzzy ART neural networks

  1999cited by this paper
• ARTMAP-IC and medical diagnosis: Instance counting and inconsistent cases

  1998cited by this paper
• A geometric approach to cluster validity for normal mixtures

  1997cited by this paper
• Biological and Artificial Computation: From Neuroscience to Technology

  1997cited by this paper
• The Kohonen Algorithm: A Powerful Tool for Analyzing and Representing Multidimensional Quantitative and Qualitative Data

  1997cited by this paper
• Gaussian ARTMAP: A Neural Network for Fast Incremental Learning of Noisy Multidimensional Maps

  1996cited by this paper
• Hierarchical clustering with ART neural networks

  1994influential reference
• An examination of procedures for determining the number of clusters in a data set

  1994cited by this paper
• Segmentation of thermal images using the fuzzy C-means algorithm

  1993cited by this paper
• Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system

  1991influential reference
• A Validity Measure for Fuzzy Clustering

  1991cited by this paper
• ART 3: Hierarchical search using chemical transmitters in self-organizing pattern recognition architectures

  1990cited by this paper
• A massively parallel architecture for a self-organizing neural pattern recognition machine

  1988cited by this paper
• The ART of adaptive pattern recognition by a self-organizing neural network

  1988cited by this paper
• ART 2: self-organization of stable category recognition codes for analog input patterns.

  1987cited by this paper
• Silhouettes: a graphical aid to the interpretation and validation of cluster analysis

  1987influential reference
• Introduction to Probability and Mathematical Statistics

  1987cited by this paper
• A Cluster Separation Measure

  1979cited by this paper
• A dendrite method for cluster analysis

  1974cited by this paper

• Meta-iCVI: Ensemble Validity Metrics for Concise Labeling of Correct, Under- or Over-Partitioning in Streaming Clustering

  2024cites this paper
• Road Crack Detection using Rooted Ratio-Dependent Scaling Factor and Pixel Difference based Fuzzy-C Means Clustering Technique

  2024cites this paper
• The Use of Digital Technologies for Flipped Learning Implementation

  2020cites this paper
• An empirical comparison of machine learning clustering methods in the study of Internet addiction among students majoring in Computer Sciences

  2019cites this paper
• The Use of Ensemble Classification and Clustering Methods of Machine Learning in the Study of Internet Addiction of Students

  year unknowncites this paper