Inference for multivariate normal mixtures

Multivariate normal mixtures provide a flexible model for high-dimensional data. They are widely used in statistical genetics, statistical finance, and other disciplines. Due to the unboundedness of the likelihood function, classical likelihood-based methods, which may have nice practical properties, are inconsistent. In this paper, we recommend a penalized likelihood method for estimating the mixing distribution. We show that the maximum penalized likelihood estimator is strongly consistent when the number of components has a known upper bound. We also explore a convenient EM-algorithm for computing the maximum penalized likelihood estimator. Extensive simulations are conducted to explore the effectiveness and the practical limitations of both the new method and the ratified maximum likelihood estimators. Guidelines are provided based on the simulation results.

• Publication year

  2008

• Venue

  Journal of Multivariate Analysis

• Publication date

  2008-05-26

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1016/j.jmva.2008.12.005arXiv 0805.3906
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• INFERENCE FOR NORMAL MIXTURES IN MEAN AND VARIANCE

  2008cited by this paper
• Variable Selection for Model-Based Clustering

  2006cited by this paper
• CONSISTENCY OF THE CONSTRAINED MAXIMUM LIKELIHOOD ESTIMATOR IN FINITE NORMAL MIXTURE MODELS

  2006cited by this paper
• Finite Mixture and Markov Switching Models

  2006cited by this paper
• Maximum Penalized Likelihood Estimation

  2006cited by this paper
• Bayesian Variable Selection in Clustering High-Dimensional Data

  2005cited by this paper
• Class discovery and classification of tumor samples using mixture modeling of gene expression data - a unified approach

  2004cited by this paper
• A likelihood-based constrained algorithm for multivariate normal mixture models

  2004cited by this paper
• Maximum Penalized Likelihood Estimation, Volume I, Density Estimation

  2004cited by this paper
• On modeling locus heterogeneity using mixture distributions

  2004cited by this paper
• Maximum penalized likelihood estimation

  2001cited by this paper
• Finite Mixture Models

  2000cited by this paper
• How Many Clusters? Which Clustering Method? Answers Via Model-Based Cluster Analysis

  1998cited by this paper
• Mixture distributions in human genetics research

  1996cited by this paper
• Mixture models : theory, geometry, and applications

  1995cited by this paper
• Multivariate Normal Mixtures: A Fast Consistent Method of Moments

  1993cited by this paper
• On Use of the EM Algorithm for Penalized Likelihood Estimation

  1990cited by this paper
• Statistical analysis of finite mixture distributions

  1986cited by this paper
• A Constrained Formulation of Maximum-Likelihood Estimation for Normal Mixture Distributions

  1985cited by this paper
• Mixture densities, maximum likelihood, and the EM algorithm

  1984cited by this paper
• ON THE CONVERGENCE PROPERTIES OF THE EM ALGORITHM

  1983cited by this paper
• Estimating the components of a mixture of normal distributions

  1969influential reference
• CONSISTENCY OF THE MAXIMUM LIKELIHOOD ESTIMATOR IN THE PRESENCE OF INFINITELY MANY INCIDENTAL PARAMETERS

  1956cited by this paper
• Theory of point estimation

  1950cited by this paper
• Note on the Consistency of the Maximum Likelihood Estimate

  1949cited by this paper
• © Institute of Mathematical Statistics, 2005 THE TOPOGRAPHY OF MULTIVARIATE NORMAL MIXTURES 1

  year unknowncited by this paper

• Latent transition analysis for longitudinal studies of post-acute infection syndromes.

  2026cites this paper
• Named Entity Swapping for Metadata Anonymization in a Text Corpus

  2025cites this paper
• All Emulators are Wrong, Many are Useful, and Some are More Useful Than Others: A Reproducible Comparison of Computer Model Surrogates

  2025cites this paper
• Mixture-Models: a one-stop Python Library for Model-based Clustering using various Mixture Models

  2024cites this paper
• Determine the Number of States in Hidden Markov Models via Marginal Likelihood

  2024cites this paper
• Byzantine-tolerant distributed learning of finite mixture models

  2024influential citation
• Avoiding inferior clusterings with misspecified Gaussian mixture models

  2023cites this paper
• Discovering Active Subspaces for High-Dimensional Computer Models

  2023cites this paper
• Identification and Estimation of Production Function with Unobserved Heterogeneity

  2023influential citation
• Testing the Number of Components in Finite Mixture Normal Regression Model with Panel Data

  2022influential citation
• Penalized maximum likelihood estimator for finite multivariate skew normal mixtures

  2022influential citation
• A note on maximum likelihood estimation for mixture models

  2022influential citation
• Variable selection in finite mixture of median regression models using skew-normal distribution

  2022cites this paper
• Discussion on “Distributional independent component analysis for diverse neuroimaging modalities” by Ben Wu, Subhadip Pal, Jian Kang, and Ying Guo

  2022cites this paper
• Robustness of Gaussian Mixture Reduction for Split-and-Conquer Learning of Finite Gaussian Mixtures

  2021cites this paper
• Minimum Wasserstein Distance Estimator under Finite Location-scale Mixtures

  2021cites this paper
• Nonparametric Bayesian Learning of Infinite Multivariate Generalized Normal Mixture Models and Its Applications

  2021cites this paper
• Classical and Bayesian Inference of a Mixture of Bivariate Exponentiated Exponential Model

  2021cites this paper
• Penalized Versus Constrained Approaches for Clusterwise Linear Regression Modeling

  2021cites this paper
• Strong consistency of the MLE under two-parameter Gamma mixture models with a structural scale parameter

  2021cites this paper
• A CLASSICAL INVARIANCE APPROACH TO THE NORMAL MIXTURE PROBLEM

  2020cites this paper
• Consistency of the MLE under a two-parameter Gamma mixture model with a structural shape parameter

  2020cites this paper
• Improved Inference of Gaussian Mixture Copula Model for Clustering and Reproducibility Analysis using Automatic Differentiation

  2020cites this paper
• Distributed Learning of Finite Gaussian Mixtures

  2020influential citation
• Bibliography

  2019cites this paper
• Introduction

  2019cites this paper
• Semi-supervised Clustering and Classification

  2019cites this paper
• Model-based Clustering: Basic Ideas

  2019cites this paper
• Model-based Clustering with Covariates

  2019cites this paper
• Non-Gaussian Model-based Clustering

  2019cites this paper
• Preface

  2019cites this paper
• Other Topics

  2019cites this paper
• Testing the Order of Multivariate Normal Mixture Models

  2019influential citation
• Discrete Data Clustering

  2019cites this paper
• Model-Based Clustering and Classification for Data Science

  2019cites this paper
• Network data

  2019cites this paper
• On consistency of the MLE under finite mixtures of location-scale distributions with a structural parameter

  2019influential citation
• Variable Selection

  2019cites this paper
• Dealing with Difficulties

  2019cites this paper
• Model-based Clustering using Automatic Differentiation: Confronting Misspecification and High-Dimensional Data

  2019cites this paper
• Model-Based Clustering and Classification for Data Science: With Applications in R

  2019cites this paper
• Model-based Classification

  2019cites this paper
• High-dimensional Data

  2019cites this paper
• A Selective Overview of Semiparametric Mixture of Regression Models

  2018cites this paper
• Bootstrap estimation and model selection for multivariate normal mixtures using parallel computing with graphics processing units

  2018cites this paper
• SEM: Structural Equation Modeling in Molecular Biology

  2018cites this paper
• Parameter estimation with profile likelihood method and penalized EM algorithm in normal mixture distributions

  2018cites this paper
• A mixture factor model with applications to microarray data

  2018cites this paper
• A simple root selection method for univariate finite normal mixture models

  2018cites this paper
• Statistical inference based on bridge divergences

  2018cites this paper
• Parameter Estimation and Multilevel Clustering with Mixture and Hierarchical Models

  2017cites this paper
• Statistical inference based on bridge divergences

  2017cites this paper
• Estimation and testing for semiparametric mixtures of partially linear models

  2017cites this paper
• On the Construction of an Automatic Traffic Sign Recognition System

  2017cites this paper
• On Implementing Probabilistic Algorithms with GPUs and its their Applications to Statistical Computation (GPU上で実行される確率アルゴリズムの研究とその 計算機統計への応用)

  2017cites this paper
• Consistency of the penalized MLE for two-parameter gamma mixture models

  2016cites this paper
• Testing for the number of states in hidden Markov models

  2016cites this paper
• Penalized Maximum Likelihood Estimator for Skew Normal Mixtures

  2016cites this paper
• Consistency of the penalized MLE for two-parameter gamma mixture models

  2016cites this paper
• MINIMAX ESTIMATION FOR MIXTURES OF

  2016cites this paper
• SNP Genotype Calling of Tetraploid Species Using Bivariate Modelling Approaches

  2016cites this paper
• Consistency of the MLE under mixture models

  2016influential citation
• RATES OF PARAMETER ESTIMATION 39

  2016cites this paper
• Warp Bridge Sampling: The Next Generation

  2016cites this paper
• Penalized maximum likelihood estimation for Gaussian hidden Markov models

  2016influential citation
• Resolving some high dimensional problems in bioinformatics

  2016cites this paper
• Principal components in the discrimination of outliers: A study in simulation sample data corrected by Pearson's and Yates´s chi-square distance

  2016cites this paper
• A data driven equivariant approach to constrained Gaussian mixture modeling

  2016cites this paper
• Convergence rates of parameter estimation for some weakly identifiable finite mixtures

  2016cites this paper
• COMPOSITE LIKELIHOOD UNDER HIDDEN MARKOV MODEL

  2015cites this paper
• Label switching and its solutions for frequentist mixture models

  2015cites this paper
• Methods in Monte Carlo Computation, Astrophysical Data Analysis and Hypothesis Testing with Multiply-Imputed Data

  2015influential citation
• Understanding and Addressing the Unbounded “Likelihood” Problem

  2015cites this paper
• Identification and estimation of hidden Markov models

  2014cites this paper
• Avoiding spurious local maximizers in mixture modeling

  2014cites this paper
• AN EXPERT-FREE TECHNIQUE FOR LIVE SITE UXO TARGET CLASSIFICATION

  2014cites this paper
• Assessment of the number of components in Gaussian mixture models in the presence of multiple local maximizers

  2014influential citation
• Automatic classification of unexploded ordnance applied to live sites for MetalMapper sensor

  2014cites this paper
• A note on the article 'Inference for multivariate normal mixtures' by J. Chen and X. Tan

  2014cites this paper
• Penalty and Barrier Methods

  2013cites this paper
• Feasibility and Duality

  2013cites this paper
• Order selection in finite mixtures of linear regressions

  2013influential citation
• Challenges in model‐based clustering

  2013cites this paper
• A mixture of generalized hyperbolic distributions

  2013cites this paper
• Consistency, Breakdown Robustness, and Algorithms for Robust Improper Maximum Likelihood Clustering

  2013cites this paper
• A Complex Approach to UXO Discrimination: Combining Advanced EMI Forward Models and Statistical Signal Processing

  2012cites this paper
• Root selection in normal mixture models

  2012influential citation
• On the upper bound of the number of modes of a multivariate normal mixture

  2012cites this paper
• Estimating Parameters in Muitivariate Normal Mixtures

  2011influential citation
• Degeneracy of the EM algorithm for the MLE of multivariate Gaussian mixtures and dynamic constraints

  2011cites this paper
• Order Selection in Finite Mixtures of Linear Regressions

  2011influential citation
• Minimax estimation for mixtures of Wishart distributions

  2011influential citation
• The Bivariate Normal Mixture Distribution: A Power Study of Bootstrap Test

  2011cites this paper
• A computational strategy for doubly smoothed MLE exemplified in the normal mixture model

  2010cites this paper
• A profile likelihood method for normal mixture with unequal variance

  2010cites this paper
• Statistical Learning in Drug Discovery via Clustering and Mixtures

  2007cites this paper
• Subject Index

  1984cites this paper
• Norges Teknisk-naturvitenskapelige Universitet Simultaneous Credible Bands for Latent Gaussian Models Simultaneous Credible Bands for Latent Gaussian Models

  year unknowncites this paper