Recovering the number of clusters in data sets with noise features using feature rescaling factors

In this paper we introduce three methods for re-scaling data sets aiming at improving the likelihood of clustering validity indexes to return the true number of spherical Gaussian clusters with additional noise features. Our method obtains feature re-scaling factors taking into account the structure of a given data set and the intuitive idea that different features may have different degrees of relevance at different clusters.We experiment with the Silhouette (using squared Euclidean, Manhattan, and the pth power of the Minkowski distance), Dunn's, Calinski-Harabasz and Hartigan indexes on data sets with spherical Gaussian clusters with and without noise features. We conclude that our methods indeed increase the chances of estimating the true number of clusters in a data set.

• Publication year

  2015

• Venue

  Information Sciences

• Publication date

  2015-12-10

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1016/j.ins.2015.06.039arXiv 1602.06989
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Finding Groups In Data

  2016cited by this paper
• On comparing partitions

  2015cited by this paper
• How to find an appropriate clustering for mixed‐type variables with application to socio‐economic stratification

  2013cited by this paper
• An extensive comparative study of cluster validity indices

  2013influential reference
• On Initializations for the Minkowski Weighted K-Means

  2012cited by this paper
• Minkowski metric, feature weighting and anomalous cluster initializing in K-Means clustering

  2012cited by this paper
• An Empirical Evaluation of Different Initializations on the Number of K-Means Iterations

  2012cited by this paper
• Clustering: A Data Recovery Approach

  2012cited by this paper
• Intelligent Choice of the Number of Clusters in K-Means Clustering: An Experimental Study with Different Cluster Spreads

  2010influential reference
• Vlfeat: an open and portable library of computer vision algorithms

  2010cited by this paper
• Data clustering: 50 years beyond K-means

  2008cited by this paper
• Experiments for the Number of Clusters in K-Means

  2007cited by this paper
• Initializing K-means Batch Clustering: A Critical Evaluation of Several Techniques

  2007cited by this paper
• Weighting Method for Feature Selection in K-Means

  2007cited by this paper
• K-means clustering: a half-century synthesis.

  2006cited by this paper
• Worst-case and Smoothed Analysis of the ICP Algorithm, with an Application to the k-means Method

  2006cited by this paper
• Automated variable weighting in k-means type clustering

  2005cited by this paper
• An optimization algorithm for clustering using weighted dissimilarity measures

  2004cited by this paper
• A Method to Identify Significant Clusters in Gene Expression Data

  2002cited by this paper
• Exploring the conditional coregulation of yeast gene expression through fuzzy k-means clustering

  2002cited by this paper
• Genesis: cluster analysis of microarray data

  2002cited by this paper
• A Method to Identify Signicant Clusters in Gene Expression Data

  2002cited by this paper
• A prediction-based resampling method for estimating the number of clusters in a dataset

  2002cited by this paper
• On Clustering Validation Techniques

  2001cited by this paper
• Some new indexes of cluster validity

  1998cited by this paper
• An examination of procedures for determining the number of clusters in a data set

  1994cited by this paper
• Algorithms for Clustering Data

  1988cited by this paper
• Silhouettes: a graphical aid to the interpretation and validation of cluster analysis

  1987cited by this paper
• A dendrite method for cluster analysis

  1974cited by this paper
• A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters

  1973cited by this paper
• A clustering technique for summarizing multivariate data.

  1967cited by this paper
• Some methods for classification and analysis of multivariate observations

  1967cited by this paper
• Some methods for classi cation and analysis of multivariate observations

  1967cited by this paper

• AI and wearable sensors in Higher Education to investigate Public Speaking Skills

  2026cites this paper
• A new multi-expert distance for clustering climate parameters: a Caribbean precipitation case study

  2026cites this paper
• A 'Cluster-then-Estimate' Natural Language Processing (NLP) Approach for Classifying Maritime Incident Severity Based on Textual Descriptions.

  2026cites this paper
• AI‐Driven Personalized Microlearning Framework for Enhanced E‐Learning

  2025cites this paper
• Determination of the Content Similarity of Text Documents Using a Two-Stage Analysis Method

  2025cites this paper
• Intelligent Clustering and Adaptive Energy Management in Wireless Sensor Networks with KDE-Based Deployment

  2025cites this paper
• Complementary operation optimization of cascade hydropower reservoirs and photovoltaic energy using cooperation search algorithm and conditional generative adversarial networks

  2025cites this paper
• AECuration: automated event curation for spike sorting

  2025cites this paper
• SLEDgeHammer: A Frequent Pattern-Based Cluster Validation Index for Categorical Data

  2025cites this paper
• Accelerating battery innovation: AI-powered molecular discovery.

  2025cites this paper
• Digital hygiene among teachers

  2025cites this paper
• Optimizing NFDRS by developing custom fuel models for wildfire risk assessment in Golestan Province, NE Iran

  2025cites this paper
• Penalized K-Means Clustering: Another Look at Its Statistical Properties

  2025cites this paper
• The Role of Curiosity in Information Dissemination on Telegram: A Case Study from the 2022 Brazilian Elections

  2025cites this paper
• Number of Clusters in a Dataset: A Regularized K-means Approach

  2025cites this paper
• Allosteric Modulation of Thrombin by Thrombomodulin: Insights from Logistic Regression and Statistical Analysis of Molecular Dynamics Simulations

  2024cites this paper
• Research on Oil and Gas-Bearing Zone Prediction and Identification Based on the SVD–K-Means Algorithm—A Case Study of the WZ6-1 Oil-Bearing Structure in the Beibu Gulf Basin, South China Sea

  2024cites this paper
• Sustainability and the social fabric in commercial apple orchards in Puebla, Mexico

  2024cites this paper
• Peak operation optimization of cascade hydropower reservoirs and solar power plants considering output forecasting uncertainty

  2024cites this paper
• System uczenia głębokiego dla eliminacji szumów z wykorzystaniem dywergencji alpha

  2024cites this paper
• A novel machine learning approach for interpolating seismic velocity and electrical resistivity models for early-stage soil-rock assessment

  2024cites this paper
• Enhanced Spectral Ensemble Clustering for Fault Diagnosis: Application to Photovoltaic Systems

  2024cites this paper
• Subsurface Lithological Characterization Via Machine Learning-assisted Electrical Resistivity and SPT-N Modeling: A Case Study from Sabah, Malaysia

  2024cites this paper
• Integration of VIS–NIR Spectroscopy and Multivariate Technique for Soils Discrimination Under Different Land Management

  2024cites this paper
• Study on urban heatwave characteristics and thermal stress scenarios based on China's heatwave hazard zoning

  2024cites this paper
• Trackside Acoustic Fault Diagnosis of Bearing Based on Doppler Knowledge Embedded in Domain Adaptation Network

  2024cites this paper
• Datacube segmentation via deep spectral clustering

  2024cites this paper
• Systematization of sunflower genotypes based on seed phenotypic characteristics using neural networks

  2024cites this paper
• Incomplete multi-view clustering based on low-rank representation with adaptive graph regularization

  2023cites this paper
• Chance-Constrained Dispatching of Integrated Energy Systems Considering Source–Load Uncertainty and Photovoltaic Absorption

  2023cites this paper
• Unraveling the Impact of W215A/E217A Mutations on Thrombin’s Dynamics and Thrombomodulin Binding through Molecular Dynamics Simulations

  2023cites this paper
• Understanding hydrologic controls of slope response to 1 precipitations through Machine Learning analysis 2 applied to synthetic data

  2023cites this paper
• A comparative analysis on prototype-based clustering methods

  2023cites this paper
• Random Feature-Based Collaborative Kernel Fuzzy Clustering for Distributed Peer-to-Peer Networks

  2023cites this paper
• Customer Segmentation Using K-Means Clustering Algorithm and RFM Model

  2023cites this paper
• Understanding the land use function of station areas based on spatiotemporal similarity in rail transit ridership: A case study in Shanghai, China

  2023cites this paper
• Deep learning of phase transitions for quantum spin chains from correlation aspects

  2023cites this paper
• Data-driven maintenance priority recommendations for civil aircraft engine fleets using reliability-based bivariate cluster analysis

  2023cites this paper
• Analysis of insoluble particles in hailstones in China

  2023cites this paper
• One-class learning for fake news detection through multimodal variational autoencoders

  2023cites this paper
• Toward Addressing Ambiguous Interactions and Inferring User Intent with Dimension Reduction and Clustering Combinations in Visual Analytics

  2023cites this paper
• Modeling of farmers' production performance portrait based on GASA-WFCM clustering

  2023cites this paper
• Discovering anomalies in big data: a review focused on the application of metaheuristics and machine learning techniques

  2023cites this paper
• Identifying the Regions of a Space with the Self-Parameterized Recursively Assessed Decomposition Algorithm (SPRADA)

  2023cites this paper
• Designing Performance Dashboard for Monitoring Post-harvest Loss in Transportation

  2023cites this paper
• Clustering and Analysis of the Driving Style in the Cut-in Process

  2023cites this paper
• Understanding hydrologic controls of sloping soil response to precipitation through machine learning analysis applied to synthetic data

  2023cites this paper
• Automatic traffic scenarios generation for autonomous ships collision avoidance system testing

  2022cites this paper
• Ship energy management system development and experimental evaluation utilizing marine loading cycles based on machine learning techniques

  2022cites this paper
• Driving Style Recognition Based on Naturalistic Driving: Volatilities, Decision-Making, and Safety Performances

  2022cites this paper
• Validating Syntactic Correctness Using Unsupervised Clustering Algorithms

  2022cites this paper
• Leveraging Structure for Improved Classification of Grouped Biased Data

  2022cites this paper
• COVID-19 Pandemic Experiences and Symptoms of Pandemic-Associated Traumatic Stress Among Mothers in the US

  2022cites this paper
• AutoML Approach to Stock Keeping Units Segmentation

  2022cites this paper
• Post hoc identification of student groups: Combining user modeling with cluster analysis

  2022cites this paper
• Parkinson’s Disease Subtyping Using Clinical Features and Biomarkers: Literature Review and Preliminary Study of Subtype Clustering

  2022cites this paper
• A fast method for discovering suitable number of clusters for fuzzy clustering

  2022cites this paper
• Comparison of deep transfer learning algorithms and transferability measures for wearable sleep staging

  2022cites this paper
• ["Reservoirs of venereal diseases," "MSM" and "PWA": continuities, ruptures and temporalities in the production of bioidentities in the context of the AIDS epidemic].

  2022cites this paper
• ECD Test: An Empirical Way based on the Cumulative Distributions to Evaluate the Number of Clusters for Unsupervised Clustering

  2022cites this paper
• ReFRS: Resource-efficient Federated Recommender System for Dynamic and Diversified User Preferences

  2022cites this paper
• Channel boundary detection using partial area effect with sub-pixel resolution

  2022cites this paper
• Model-based clustering (MBC) for road data via multivariate mixture of normal distributions and factor analysis (FA)

  2022cites this paper
• Caracterización de la vegetación en sitios con evidencias de antropización en Delta Frontal del Río Paraná

  2022cites this paper
• Incoherent Synthesis of Sparse Broadband Arrays based on a Parameter-Free Subspace Clustering

  2022cites this paper
• Research profile clusters among lecturers in non-traditional higher education. An exploratory analysis in the Swiss context

  2022cites this paper
• Projected fuzzy C-means with probabilistic neighbors

  2022cites this paper
• k-means clustering for persistent homology

  2022cites this paper
• Correlation between corneal dynamic responses and keratoconus topographic parameters

  2022cites this paper
• Spatiotemporal bias adjustment of IMERG satellite precipitation data across Canada

  2022cites this paper
• Application of a Deep Learning Network for Joint Prediction of Associated Fluid Production in Unconventional Hydrocarbon Development

  2022cites this paper
• Magnetism in four-layered Aurivillius Bi<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" id="d1e116" altimg="si4.svg"><mml:msub><mml:mrow /><mml:mrow><mml:mn>5</mml:mn></mml:mrow></mml:msub></mml:math>FeTi<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inli

  2022cites this paper
• Critical Evaluation of Building Defects Research: A Scientometric Analysis

  2022cites this paper
• Simulations suggest double sodium binding induces unexpected conformational changes in thrombin

  2022cites this paper
• Metrics of social curiosity: The WhatsApp case

  2022cites this paper
• Rationale and performances of a data-driven method for computing the duration of pharmacological prescriptions using secondary data sources

  2022cites this paper
• An Efficient Power System Planning Model Considering Year-Round Hourly Operation Simulation

  2022cites this paper
• Robust deep learning early alarm prediction model based on the behavioural smell for android malware

  2022cites this paper
• Grid-DPC: Improved density peaks clustering based on spatial grid walk

  2022cites this paper
• Assessing the hotspots of crop water footprint in Jilin Province of China

  2022cites this paper
• Maritime traffic clustering to capture high-risk multi-ship encounters in complex waters

  2022cites this paper
• A hybrid-learning decomposition algorithm for competing risk identification within fleets of complex engineering systems

  2022cites this paper
• Worry, coping and resignation - A repeated-measures study on emotional responses after a year in the pandemic

  2021cites this paper
• Role of AI and AI-Derived Techniques in Brain and Behavior Computing

  2021cites this paper
• Application of Cluster Analysis for the Study of Factors Affecting the Rating of Schools in Moscow

  2021cites this paper
• Grid-Based Whole Trajectory Clustering in Road Networks Environment

  2021cites this paper
• A machine learning-based clustering approach to diagnose multi-component degradation of aircraft fuel systems

  2021cites this paper
• Interpreting clinical latent representations using autoencoders and probabilistic models

  2021cites this paper
• Optical characterisation and classification of water types in the southern South China Sea and Straits of Malacca

  2021cites this paper
• Estimating the number of clusters in a ranking data context

  2021cites this paper
• Rail Steel Health Analysis Based on a Novel Genetic Density-based Clustering Technique and Manifold Representation of Acoustic Emission Signals

  2021cites this paper
• Causality Distance Measures for Multivariate Time Series with Applications

  2021cites this paper
• A Hybrid Framework for Detecting and Eliminating Cyber-Attacks in Power Grids

  2021cites this paper
• Deep learning of topological phase transitions from entanglement aspects: An unsupervised way

  2021cites this paper
• INFLUENCE OF DISTRIBUTION GRID CHARACTERISTICS ON REDISPATCH POTENTIAL OF DISPERSED FLEXIBILITY

  2021cites this paper
• Driving performance grading and analytics: learning internal indicators and external factors from multi-source data

  2021cites this paper
• Clustering high‐frequency financial time series based on information theory

  2021cites this paper
• An aerosol classification scheme for global simulations using the K-means machine learning method

  2021cites this paper
• Module Failure Feature Detection by Cluster Analysis for Fleets of Civil Aircraft Engines

  2021cites this paper
• Digital Divide: Mapping the geodemographics of internet accessibility across Great Britain

  2021cites this paper