Computational aspects of fitting mixture models via the expectation-maximization algorithm

The Expectation-Maximization (EM) algorithm is a popular tool in a wide variety of statistical settings, in particular in the maximum likelihood estimation of parameters when clustering using mixture models. A serious pitfall is that in the case of a multimodal likelihood function the algorithm may become trapped at a local maximum, resulting in an inferior clustering solution. In addition, convergence to an optimal solution can be very slow. Methods are proposed to address these issues: optimizing starting values for the algorithm and targeting maximization steps efficiently. It is demonstrated that these approaches can produce superior outcomes to initialization via random starts or hierarchical clustering and that the rate of convergence to an optimal solution can be greatly improved.

• Publication year

  2012

• Venue

  Computational Statistics & Data Analysis

• Publication date

  2012-12-01

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1016/j.csda.2012.05.011
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• R: A language and environment for statistical computing.

  2014cited by this paper
• Model-based classification via mixtures of multivariate t-distributions

  2011cited by this paper
• On the Bumpy Road to the Dominant Mode

  2010cited by this paper
• Simulating Data to Study Performance of Finite Mixture Modeling and Clustering Algorithms

  2010cited by this paper
• Serial and parallel implementations of model-based clustering via parsimonious Gaussian mixture models

  2010cited by this paper
• Finite Mixture Models

  2008cited by this paper
• Advances in Mixture Models

  2007cited by this paper
• An Introduction to the Bootstrap

  2007cited by this paper
• Mixtures of Exponential Distributions

  2006cited by this paper
• Basic Statistics and Data Analysis

  2005cited by this paper
• Types of likelihood maxima in mixture models and their implication on the performance of tests

  2004cited by this paper
• Breakdown points for maximum likelihood estimators of location–scale mixtures

  2004cited by this paper
• All of Statistics

  2004cited by this paper
• Choosing starting values for the EM algorithm for getting the highest likelihood in multivariate Gaussian mixture models

  2003influential reference
• Editorial: recent developments in mixture models

  2003cited by this paper
• Choosing initial values for the EM algorithm for finite mixtures

  2003cited by this paper
• MCLUST: Software for Model-Based Clustering, Density Estimation and Discriminant Analysis

  2002cited by this paper
• MCLUST: Software for Model-Based Cluster Analysis

  1999cited by this paper
• A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants

  1998cited by this paper
• Efficient Approximations for the Marginal Likelihood of Bayesian Networks with Hidden Variables

  1997cited by this paper
• Modelling the distribution of stamp paper thickness via finite normal mixtures: The 1872 Hidalgo stamp issue of Mexico revisited

  1997cited by this paper
• The EM algorithm and extensions

  1996influential reference
• A hybrid EM/Gauss-Newton algorithm for maximum likelihood in mixture distributions

  1996cited by this paper
• Regularized Gaussian Discriminant Analysis through Eigenvalue Decomposition

  1996cited by this paper
• An Introduction to the Bootstrap

  1995cited by this paper
• Gaussian parsimonious clustering models

  1995cited by this paper
• An Introduction to Regression Graphics

  1995cited by this paper
• Mixture models : theory, geometry, and applications

  1995cited by this paper
• The distribution of the likelihood ratio for mixtures of densities from the one-parameter exponential family

  1994cited by this paper
• Model-based Gaussian and non-Gaussian clustering

  1993cited by this paper
• Maximum likelihood estimation via the ECM algorithm: A general framework

  1993cited by this paper
• Density estimation with confidence sets exemplified by superclusters and voids in the galaxies

  1990cited by this paper
• Philatelic Mixtures and Multimodal Densities

  1988cited by this paper
• Likelihood Estimation with Normal Mixture Models

  1985cited by this paper
• Mixture densities, maximum likelihood, and the EM algorithm

  1984cited by this paper
• Maximum Likelihood Estimation of the Parameters in a Mixture of Two Univariate Normal Distributions; a Comparison of Different Algorithms

  1984cited by this paper
• Finite Mixture Distributions

  1982cited by this paper
• 9 The classification and mixture maximum likelihood approaches to cluster analysis

  1982cited by this paper
• Estimating the Dimension of a Model

  1978influential reference
• Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper

  1977cited by this paper
• THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS

  1936cited by this paper
• XXV.—On Bernoulli's Numerical Solution of Algebraic Equations

  year unknowninfluential reference
• © Institute of Mathematical Statistics, 2005 THE TOPOGRAPHY OF MULTIVARIATE NORMAL MIXTURES 1

  year unknowncited by this paper

• Disentangling Qualitatively Different Faking Strategies in High-Stakes Personality Assessments: A Mixture Extension of the Multidimensional Nominal Response Model

  2025cites this paper
• Challenges and Adaptations of Model‐Based Clustering for Flow and Mass Cytometry

  2025cites this paper
• Accelerated distributed expectation-maximization algorithms for the parameter estimation in multivariate Gaussian mixture models

  2024cites this paper
• A practical approach to making use of uncertain species presence-only data in ecology: Reclassification, regularization methods and observer bias

  2023cites this paper
• Estimation of reinforced urn processes under left-truncation and right-censoring

  2023cites this paper
• Distributed online expectation-maximization algorithm for Poisson mixture model

  2023cites this paper
• Estimation of 5G Core and RAN End-to-End Delay through Gaussian Mixture Models

  2022cites this paper
• How to make use of unlabeled observations in species distribution modeling using point process models

  2021cites this paper
• An IOHMM-Based Framework to Investigate Drift in Effectiveness of IoT-Based Systems

  2021cites this paper
• Information geometry approach to parameter estimation in hidden Markov model

  2021cites this paper
• of the Bernoulli Society for Mathematical Statistics and Probability Volume Twenty Eight Number One February 2022

  2021cites this paper
• A Generic Clustering-Based Algorithm for Approximating IOHMM Topology and Parameters

  2021cites this paper
• LiMIIRL: Lightweight Multiple-Intent Inverse Reinforcement Learning

  2021cites this paper
• The Double Flexible Dirichlet: A Structured Mixture Model for Compositional Data

  2021cites this paper
• A stochastic block model for interaction lengths

  2020cites this paper
• Model-based Classification

  2019cites this paper
• Network data

  2019cites this paper
• A dynamic stochastic blockmodel for interaction lengths

  2019cites this paper
• Volatile organic chemical analyser (eNose) diagnosis of dogs naturally infected with visceral leishmaniasis in Brazil

  2019cites this paper
• Variable Selection

  2019cites this paper
• Classification of unlabeled observations in Species Distribution Modelling using Point Process Models

  2019cites this paper
• Dealing with Difficulties

  2019cites this paper
• Finite sample properties of parametric MMD estimation: robustness to misspecification and dependence

  2019cites this paper
• Discrete Data Clustering

  2019cites this paper
• Preface

  2019cites this paper
• Non-Gaussian Model-based Clustering

  2019cites this paper
• Model-based Clustering with Covariates

  2019cites this paper
• Model-based Clustering: Basic Ideas

  2019cites this paper
• Semi-supervised Clustering and Classification

  2019cites this paper
• Introduction

  2019cites this paper
• Bibliography

  2019cites this paper
• Model-Based Clustering and Classification for Data Science: With Applications in R

  2019cites this paper
• High-dimensional Data

  2019cites this paper
• Other Topics

  2019cites this paper
• Model-Based Clustering and Classification for Data Science

  2019cites this paper
• Classification Performances Of Data Mining Clustering Algorithms For Remotely Sensed Multispectral Image Data

  2018cites this paper
• Consistency of variational Bayes inference for estimation and model selection in mixtures

  2018cites this paper
• Variable selection methods for model-based clustering

  2017influential citation
• Eigenvalues and constraints in mixture modeling: geometric and computational issues

  2017cites this paper
• A structured Dirichlet mixture model for compositional data: inferential and applicative issues

  2016cites this paper
• Regression analysis of correlated circular data based on the multivariate von Mises distribution

  2016cites this paper
• Monitoring Wireless Networks Through Machine Learning Algorithms

  2015cites this paper
• Model based clustering for mixed data: clustMD

  2015cites this paper
• Improved model-based clustering performance using Bayesian initialization averaging

  2015influential citation
• Initializing EM algorithm for univariate Gaussian, multi-component, heteroscedastic mixture models by dynamic programming partitions

  2015cites this paper
• Retrospective Bayesian outlier detection in INGARCH series

  2015cites this paper
• Multivariate methods using mixtures: Correspondence analysis, scaling and pattern-detection

  2014cites this paper
• Paired Indices for Clustering Evaluation - Correction for Agreement by Chance

  2014cites this paper
• A multivariate linear regression analysis using finite mixtures of t distributions

  2014cites this paper
• Model-based clustering of high-dimensional data: A review

  2014cites this paper
• A Mixture Model for Longitudinal Trajectories

  2014cites this paper
• Cox proportional hazards models with frailty for negatively correlated employment processes

  2014cites this paper
• Finite mixtures of multivariate skew t-distributions: some recent and new results

  2014cites this paper
• Retrospective Bayesian outlier detection in INGARCH series

  2013cites this paper
• Extending mixtures of factor models using the restricted multivariate skew-normal distribution

  2013cites this paper
• Probabilistic model‐based discriminant analysis and clustering methods in chemometrics

  2013cites this paper
• Flexible mixture modelling using the multivariate skew-t-normal distribution

  2013cites this paper
• Probabilistic model‐based discriminant analysis and clustering methods in chemometrics

  2013cites this paper
• Bayesian outlier detection in INGARCH time series

  2012cites this paper
• SFB 823 Bayesian outlier detection in INGARCH time series D iscussion P aper

  2012cites this paper
• Report of President 2011/2012 - School of Mathematical Sciences

  2012cites this paper
• Finite mixtures of multivariate skew t-distributions: some recent and new results

  2012cites this paper
• Subject Index

  1984cites this paper
• Title Model Based Clustering for Mixed Data : clustMD

  year unknowncites this paper
• Computational Statistics and Data Analysis Discretization-based Direct Random Sample Generation

  year unknowncites this paper