Simultaneous model-based clustering and visualization in the Fisher discriminative subspace

Clustering in high-dimensional spaces is nowadays a recurrent problem in many scientific domains but remains a difficult task from both the clustering accuracy and the result understanding points of view. This paper presents a discriminative latent mixture (DLM) model which fits the data in a latent orthonormal discriminative subspace with an intrinsic dimension lower than the dimension of the original space. By constraining model parameters within and between groups, a family of 12 parsimonious DLM models is exhibited which allows to fit onto various situations. An estimation algorithm, called the Fisher-EM algorithm, is also proposed for estimating both the mixture parameters and the discriminative subspace. Experiments on simulated and real datasets highlight the good performance of the proposed approach as compared to existing clustering methods while providing a useful representation of the clustered data. The method is as well applied to the clustering of mass spectrometry data.

• Publication year

  2011

• Venue

  Statistics and computing

• Publication date

  2011-01-12

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1007/s11222-011-9249-9arXiv 1101.2374
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Pattern Classification

  2012cited by this paper
• Principal Curves

  2010cited by this paper
• Heteroscedastic factor mixture analysis

  2010cited by this paper
• A Hybrid Feature Extraction Selection Approach for High-Dimensional Non-Gaussian Data Clustering

  2009cited by this paper
• Variable Selection for Clustering with Gaussian Mixture Models

  2009cited by this paper
• A Flexible and Efficient Algorithm for Regularized Fisher Discriminant Analysis

  2009cited by this paper
• Biomarker discovery in MALDI-TOF serum protein profiles using discrete wavelet transformation

  2009cited by this paper
• Finite Mixture Models

  2008cited by this paper
• Parsimonious Gaussian mixture models

  2008cited by this paper
• Discriminative K-means for Clustering

  2007cited by this paper
• DALASS: Variable selection in discriminant analysis via the LASSO

  2007cited by this paper
• Adaptive dimension reduction using discriminant analysis and K-means clustering

  2007cited by this paper
• Discriminative cluster analysis

  2006cited by this paper
• Introduction to Statistical Pattern Recognition

  2006cited by this paper
• Variable Selection for Model-Based Clustering

  2006cited by this paper
• High-dimensional data clustering

  2006cited by this paper
• Characterization of a Family of Algorithms for Generalized Discriminant Analysis on Undersampled Problems

  2005influential reference
• High-Dimensional Discriminant Analysis

  2005influential reference
• The Curse of Dimensionality in Data Mining and Time Series Prediction

  2005cited by this paper
• Generalizing discriminant analysis using the generalized singular value decomposition

  2004cited by this paper
• Subspace clustering for high dimensional data: a review

  2004cited by this paper
• Simultaneous feature selection and clustering using mixture models

  2004cited by this paper
• A generalized Foley-Sammon transform based on generalized fisher discriminant criterion and its application to face recognition

  2003cited by this paper
• Principal Component Analysis

  2003cited by this paper
• Modelling high-dimensional data by mixtures of factor analyzers

  2003cited by this paper
• Choosing starting values for the EM algorithm for getting the highest likelihood in multivariate Gaussian mixture models

  2003cited by this paper
• Model-Based Clustering, Discriminant Analysis, and Density Estimation

  2002cited by this paper
• K-means Iterative Fisher (KIF) unsupervised clustering algorithm applied to image texture segmentation

  2002cited by this paper
• Principles of Multivariate Analysis

  2001cited by this paper
• A theorem on the uncorrelated optimal discriminant vectors

  2001cited by this paper
• The Elements of Statistical Learning: Data Mining, Inference, and Prediction

  2001cited by this paper
• A nonlinear PCA based on manifold approximation

  2000cited by this paper
• Finite Mixture Models

  2000cited by this paper
• Assessing a Mixture Model for Clustering with the Integrated Completed Likelihood

  2000influential reference
• MCLUST: Software for Model-Based Cluster Analysis

  1999influential reference
• Mixtures of Probabilistic Principal Component Analyzers

  1999influential reference
• Data clustering: a review

  1999cited by this paper
• GTM: The Generative Topographic Mapping

  1998cited by this paper
• Nonlinear Component Analysis as a Kernel Eigenvalue Problem

  1998cited by this paper
• Automatic subspace clustering of high dimensional data for data mining applications

  1998influential reference
• Dimension Reduction by Local Principal Component Analysis

  1997cited by this paper
• The EM algorithm and extensions

  1996cited by this paper
• Penalized Discriminant Analysis

  1995cited by this paper
• Dynamic Programming

  1993cited by this paper
• A generalized optimal set of discriminant vectors

  1992cited by this paper
• A Classification EM algorithm for clustering and two stochastic versions

  1992cited by this paper
• A note on the orthonormal discriminant vector method for feature extraction

  1991cited by this paper
• Introduction to statistical pattern recognition (2nd ed.)

  1990cited by this paper
• Regularized Discriminant Analysis

  1989cited by this paper
• CANONICAL VARIATE ANALYSIS—A GENERAL MODEL FORMULATION

  1984cited by this paper
• Matrix computations

  1983cited by this paper
• EM algorithms for ML factor analysis

  1982cited by this paper
• Estimating the Dimension of a Model

  1978influential reference
• Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper

  1977cited by this paper
• An Optimal Set of Discriminant Vectors

  1975cited by this paper
• A new look at the statistical model identification

  1974cited by this paper
• Pattern Classification

  1973cited by this paper
• Some results on Tchebycheffian spline functions

  1971cited by this paper
• THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS

  1936influential reference
• Transactions on Pattern Analysis and Machine Intelligence 1 Mixtures of Factor Analyzers with Common Factor Loadings: Applications to the Clustering and Visualisation of High-dimensional Data

  year unknowninfluential reference

• Galaxy Evolution with Manifold Learning

  2026cites this paper
• The influence of linear and nonlinear pedagogies on movement exploration of a weightlifting skill.

  2025cites this paper
• A Tutorial on Discriminative Clustering and Mutual Information

  2025cites this paper
• Stacking Model‐Based Classifiers for Dealing With Multiple Sets of Noisy Labels

  2025cites this paper
• Mixture models for simultaneous classification and reduction of three-way data

  2024influential citation
• Deciphering the Feature Representation of Deep Neural Networks for High-Performance AI

  2024cites this paper
• Limitations of clustering with PCA and correlated noise

  2024cites this paper
• Measurement and Contagion Modelling of Systemic Risk in China's Financial Sectors: Evidence for Functional Data Analysis and Complex Network

  2023cites this paper
• Species of Dickinsonia Sprigg from the Ediacaran of South Australia

  2023cites this paper
• Assessment of Excess Mortality in Italy in 2020–2021 as a Function of Selected Macro-Factors

  2023cites this paper
• Narrowing the coordination solution space during motor learning standardizes individual patterns of search strategy but diversifies learning rates

  2023cites this paper
• Sparse and geometry-aware generalisation of the mutual information for joint discriminative clustering and feature selection

  2023cites this paper
• Sparse GEMINI for Joint Discriminative Clustering and Feature Selection

  2023cites this paper
• Different pedagogical approaches to motor imagery both demonstrate individualized movement patterns to achieve improved performance outcomes when learning a complex motor skill

  2023cites this paper
• Model-Based Clustering with Nested Gaussian Clusters

  2023cites this paper
• Finite mixtures of multivariate Poisson-log normal factor analyzers for clustering count data

  2023cites this paper
• Discriminative projection fuzzy K-Means with adaptive neighbors

  2023cites this paper
• Statistical indicators based on mobile phone and street maps data for risk management in small urban areas

  2023cites this paper
• Cluster Analysis for the Selection of Potential Discriminatory Variables and the Identification of Subgroups in Archaeometry

  2023cites this paper
• An Ensemble Framework to Improve the Accuracy of Prediction Using Clustered Random-Forest and Shrinkage Methods

  2022cites this paper
• Is Prescription of Specific Movement Form Necessary for Optimal Skill Development? A Nonlinear Pedagogy Approach

  2022cites this paper
• Bilingualism effects on cognition in autistic children are not all-or-nothing: The role of socioeconomic status in intellectual skills in bilingual autistic children

  2022cites this paper
• Unsupervised classification reveals new evolutionary pathways

  2022cites this paper
• Clustering analysis of movement kinematics in reinforcement learning

  2021cites this paper
• Sparse mixture of von Mises-Fisher distribution

  2021cites this paper
• Logistic Normal Multinomial Factor Analyzers for Clustering Microbiome Data

  2021cites this paper
• Noise-insensitive discriminative subspace fuzzy clustering

  2021cites this paper
• Model-Based Clustering, Classification, and Discriminant Analysis Using the Generalized Hyperbolic Distribution: MixGHD R package

  2021cites this paper
• Perceptual-motor skill transfer: Multidimensionality and specificity of both general and specific transfers.

  2021cites this paper
• Group-Wise Shrinkage Estimation in Penalized Model-Based Clustering

  2021cites this paper
• Unsupervised classification of SDSS galaxy spectra

  2021cites this paper
• Synergies between low- and intermediate-redshift galaxy populations revealed with unsupervised machine learning

  2021influential citation
• Model based clustering using finite mixtures of multivariate geometric skew normal distribution

  2020cites this paper
• Clustering of Time-Series Data

  2020cites this paper
• A parsimonious family of multivariate Poisson-lognormal distributions for clustering multivariate count data

  2020cites this paper
• l2,p-Norm Based Discriminant Subspace Clustering Algorithm

  2020cites this paper
• Multi-Partitions Subspace Clustering

  2020influential citation
• Improved subspace clustering algorithm using multi-objective framework and subspace optimization

  2020cites this paper
• Statistical Analysis of Microarray Data Clustering using NMF, Spectral Clustering, Kmeans, and GMM

  2020cites this paper
• A family of parsimonious mixtures of multivariate Poisson‐lognormal distributions for clustering multivariate count data

  2020cites this paper
• A Bayesian Fisher-EM algorithm for discriminative Gaussian subspace clustering

  2020influential citation
• Model-Based Clustering and Classification for Data Science

  2019cites this paper
• Other Topics

  2019cites this paper
• Discrete Data Clustering

  2019cites this paper
• High-dimensional Data

  2019cites this paper
• Model-Based Clustering and Classification for Data Science: With Applications in R

  2019cites this paper
• Bibliography

  2019cites this paper
• Variable Selection

  2019cites this paper
• Model-based Clustering with Covariates

  2019cites this paper
• Introduction

  2019cites this paper
• Network data

  2019cites this paper
• Fault Diagnosis in Chemical Processes Based on Class-Incremental FDA and PCA

  2019cites this paper
• Non-Gaussian Model-based Clustering

  2019cites this paper
• Preface

  2019cites this paper
• Automatic classification of sources in large astronomical catalogs

  2019cites this paper
• Semi-supervised Clustering and Classification

  2019cites this paper
• Model-based Classification

  2019cites this paper
• Model-based Clustering: Basic Ideas

  2019cites this paper
• Dealing with Difficulties

  2019cites this paper
• The VIMOS Public Extragalactic Redshift Survey (VIPERS)

  2018cites this paper
• The VIMOS Public Extragalactic Redshift Survey (VIPERS). Unsupervised classification with photometric redshifts: a method to accurately classify large galaxy samples without spectroscopic information

  2018cites this paper
• Reduced-Rank Modeling for High-Dimensional Model-Based Clustering

  2018cites this paper
• Distance Metrics and Clustering Methods for Mixed‐type Data

  2018cites this paper
• Variable Selection for Skewed Model-Based Clustering: Application to the Identification of Novel Sleep Phenotypes

  2018cites this paper
• Efficient discriminative clustering via QR decomposition-based Linear Discriminant Analysis

  2018cites this paper
• Bayesian inference by reversible jump MCMC for clustering based on finite generalized inverted Dirichlet mixtures

  2018cites this paper
• Between exploitation and exploration of motor behaviours: unpacking the constraints-led approach to foster nonlinear learning in physical education

  2018cites this paper
• Multiway-Based Weigh-in-Motion Data-Clustering Analysis for Pavement ME Design

  2017cites this paper
• Unsupervised classification in high dimension

  2017cites this paper
• A Model-Based Approach to Simultaneous Clustering and Dimensional Reduction of Ordinal Data

  2017cites this paper
• A survey of feature selection methods for Gaussian mixture models and hidden Markov models

  2017cites this paper
• Extending Growth Mixture Models Using Continuous Non-Elliptical Distributions

  2017cites this paper
• Model selection for sparse high-dimensional learning

  2017cites this paper
• Variable selection methods for model-based clustering

  2017cites this paper
• Model-based clustering with sparse covariance matrices

  2017cites this paper
• THE DISCRIMINATIVE FUNCTIONAL MIXTURE MODEL FOR A COMPARATIVE ANALYSIS OF BIKE SHARING SYSTEMS

  2016influential citation
• Behavioural variability and motor performance: Effect of practice specialization in front crawl swimming.

  2016cites this paper
• Dimension Reduction in Clustering

  2016cites this paper
• rCOSA: A Software Package for Clustering Objects on Subsets of Attributes

  2016cites this paper
• Model-based clustering of high-dimensional data in Astrophysics

  2016influential citation
• The discriminative functional mixture model for a comparative analysis of bike sharing systems

  2016influential citation
• What Variability tells us about motor expertise: measurements and perspectives from a complex system approach

  2015cites this paper
• A classification method for binary predictors combining similarity measures and mixture models

  2015cites this paper
• Sparse optimal discriminant clustering

  2015cites this paper
• Factor probabilistic distance clustering (FPDC): a new clustering method

  2015cites this paper
• A pairwise likelihood approach to simultaneous clustering and dimensional reduction of ordinal data

  2015cites this paper
• Product selection for liking studies: The sensory informed design

  2015cites this paper
• Modeling industrial loads in non-residential buildings

  2015cites this paper
• Détection d'anomalies à la volée dans des flux de données de grande dimension

  2014cites this paper
• Dimension Reduction and Clustering using Non-Elliptical Mixtures

  2014influential citation
• Unsupervised learning via mixtures of skewed distributions with hypercube contours

  2014cites this paper
• Model-based clustering of high-dimensional data: A review

  2014influential citation
• Optimized Unsupervised Image Classification Based on Neutrosophic Set Theory

  2014cites this paper
• Key Point Selection and Clustering of Swimmer Coordination Through Sparse Fisher-EM

  2014influential citation
• The Discriminative Functional Mixture Model for the Analysis of Bike Sharing Systems

  2014cites this paper
• Energy use forecast and model predictive control of building complexes

  2014cites this paper
• Dimension reduction for model-based clustering via mixtures of multivariate $$t$$t-distributions

  2013cites this paper
• Learning algorithms for sparse classification

  2013cites this paper
• Recent developments in expectation‐maximization methods for analyzing complex data

  2013cites this paper
• Probabilistic model‐based discriminant analysis and clustering methods in chemometrics

  2013influential citation