Paired Indices for Clustering Evaluation - Correction for Agreement by Chance

In the present paper we focus on the performance of clustering algorithms recurring to indices of paired agreement to measure the accordance between clusters and an a priori known structure. We specifically propose a method to correct all indices considered for agreement by chance – the adjusted indices are meant to provide a realistic measure of clustering performance. The proposed method enables the correction of virtually any index – overcoming previous limitations known in the literature - and provides very precise results. We use simulated datasets under diverse scenarios and discuss the pertinence of our proposal which is particularly relevant when poorly separated clusters are considered. Finally we compare the performance of EM and K-Means algorithms, within each of the simulated scenarios and generally conclude that EM generally yields best results.

• Publication year

  2014

• Venue

  International Conference on Enterprise Information Systems

• Publication date

  2014-04-27

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.5220/0004868301640170
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Rmixmod: The R Package of the Model-Based Unsupervised, Supervised and Semi-Supervised Classification Mixmod Library

  2015cited by this paper
• Clustering cross-validation and mutual information indices

  2012cited by this paper
• Computational aspects of fitting mixture models via the expectation-maximization algorithm

  2012cited by this paper
• Correcting Jaccard and other similarity indices for chance agreement in cluster analysis

  2011cited by this paper
• Stability and model selection in k-means clustering

  2010cited by this paper
• Means and variances for a family of similarity indices used in cluster analysis

  2010influential reference
• A SOM-Based Document Clustering Using Frequent Max Substrings for Non-Segmented Texts

  2010cited by this paper
• Simulating Data to Study Performance of Finite Mixture Modeling and Clustering Algorithms

  2010influential reference
• Data clustering: 50 years beyond K-means

  2008cited by this paper
• Cluster-wise assessment of cluster stability

  2007cited by this paper
• On Similarity Indices and Correction for Chance Agreement

  2006cited by this paper
• Comparison of similarity coefficients used for cluster analysis with dominant markers in maize (Zea mays L)

  2004cited by this paper
• A Study of the Comparability of External Criteria for Hierarchical Cluster Analysis.

  1986cited by this paper
• Metric and Euclidean properties of dissimilarity coefficients

  1986cited by this paper
• Comparing partitions

  1985cited by this paper
• A Method for Comparing Two Hierarchical Clusterings

  1983cited by this paper
• Exact conditional tests for cross-classifications: Approximation of attained significance levels

  1979cited by this paper
• Measures of association for cross classifications

  1979cited by this paper
• Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper

  1977cited by this paper
• Natural Scales in Geographical Patterns

  1971cited by this paper
• A rigorous derivation of the exact contingency formula

  1969cited by this paper
• Principles of numerical taxonomy

  1965influential reference
• On Habitat and Association of Species of Anopheline Larvae in South-eastern Madras.

  1940cited by this paper

• No citing papers are available for this paper.