Identifying Mixtures of Mixtures Using Bayesian Estimation

ABSTRACT The use of a finite mixture of normal distributions in model-based clustering allows us to capture non-Gaussian data clusters. However, identifying the clusters from the normal components is challenging and in general either achieved by imposing constraints on the model or by using post-processing procedures. Within the Bayesian framework, we propose a different approach based on sparse finite mixtures to achieve identifiability. We specify a hierarchical prior, where the hyperparameters are carefully selected such that they are reflective of the cluster structure aimed at. In addition, this prior allows us to estimate the model using standard MCMC sampling methods. In combination with a post-processing approach which resolves the label switching issue and results in an identified model, our approach allows us to simultaneously (1) determine the number of clusters, (2) flexibly approximate the cluster distributions in a semiparametric way using finite mixtures of normals and (3) identify cluster-specific parameters and classify observations. The proposed approach is illustrated in two simulation studies and on benchmark datasets. Supplementary materials for this article are available online.

• Publication year

  2015

• Venue

  Journal of Computational And Graphical Statistics

• Publication date

  2015-02-23

• Fields of study

  Mathematics, Computer Science, Medicine

• Identifiers
  DOI 10.1080/10618600.2016.1200472arXiv 1502.06449PMID 28626349PMCID 5455957
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Merging Mixture Components for Clustering Through Pairwise Overlap

  2016cited by this paper
• On comparing partitions

  2015influential reference
• Finite mixtures of multivariate skew t-distributions: some recent and new results

  2014cited by this paper
• The Infinite Mixture of Infinite Gaussian Mixtures

  2014cited by this paper
• A “Density-Based” Algorithm for Cluster Analysis Using Species Sampling Gaussian Mixture Models

  2014cited by this paper
• R: A language and environment for statistical computing.

  2014cited by this paper
• Model-based clustering based on sparse finite Gaussian mixtures

  2014cited by this paper
• Parsimonious skew mixture models for model-based clustering and classification

  2013cited by this paper
• Hierarchical Modeling for Rare Event Detection and Cell Subset Alignment across Flow Cytometry Samples

  2013cited by this paper
• PReMiuM: An R Package for Profile Regression Mixture Models Using Dirichlet Processes.

  2013influential reference
• Hierarchical Bayesian mixture modelling for antigen-specific T-cell subtyping in combinatorially encoded flow cytometry studies

  2013cited by this paper
• Bayesian Nonparametric Inference - Why and How.

  2013cited by this paper
• Author ' s personal copy 100

  2013influential reference
• Model-based clustering and classification with non-normal mixture distributions

  2013cited by this paper
• Parsimonious Shifted Asymmetric Laplace Mixtures

  2013cited by this paper
• Finite mixtures of multivariate skew t-distributions: some recent and new results

  2012cited by this paper
• Mixtures of Shifted AsymmetricLaplace Distributions

  2012cited by this paper
• EMMIXuskew: An R Package for Fitting Mixtures of Multivariate Skew t Distributions via the EM Algorithm

  2012cited by this paper
• mclust Version 4 for R : Normal Mixture Modeling for Model-Based Clustering , Classification , and Density Estimation

  2012cited by this paper
• Mixtures of Shifted Asymmetric Laplace Distributions

  2012influential reference
• Panel data analysis: a survey on model-based clustering of time series

  2011cited by this paper
• DPpackage: Bayesian Semi- and Nonparametric Modeling in R

  2011cited by this paper
• Asymptotic behaviour of the posterior distribution in overfitted mixture models

  2011influential reference
• Inference with normal-gamma prior distributions in regression problems

  2010cited by this paper
• Understanding GPU Programming for Statistical Computation: Studies in Massively Parallel Massive Mixtures

  2010cited by this paper
• Combining Mixture Components for Clustering

  2010cited by this paper
• Methods for merging Gaussian mixture components

  2010cited by this paper
• Bayesian profile regression with an application to the National Survey of Children's Health.

  2010cited by this paper
• Optimization of a highly standardized carboxyfluorescein succinimidyl ester flow cytometry panel and gating strategy design using discriminative information measure evaluation

  2010cited by this paper
• Dimensionally Reduced Model-Based Clustering Through Mixtures of Factor Mixture Analyzers

  2010cited by this paper
• Bayesian inference for finite mixtures of univariate and multivariate skew-normal and skew-t distributions.

  2010influential reference
• The Nested Dirichlet Process

  2008influential reference
• Statistical mixture modeling for cell subtype identification in flow cytometry

  2008cited by this paper
• Bayesian Model-Based Clustering Procedures

  2007cited by this paper
• High-content flow cytometry and temporal data analysis for defining a cellular signature of graft-versus-host disease.

  2007cited by this paper
• A mixture of mixture models for a classification problem: The unity measure error

  2007cited by this paper
• Bayesian Inference for Gene Expression and Proteomics: Model-Based Clustering for Expression Data via a Dirichlet Process Mixture Model

  2006cited by this paper
• 10 Model-Based Clustering for Expression Data via a Dirichlet Process Mixture Model

  2006cited by this paper
• Finite Mixture and Markov Switching Models

  2006influential reference
• Hierarchical Dirichlet Processes

  2006influential reference
• Clustering Univariate Observations via Mixtures of Unimodal Normal Mixtures

  2005cited by this paper
• Clustering Based on a Multilayer Mixture Model

  2005cited by this paper
• Estimating Marginal Likelihoods for Mixture and Markov Switching Models Using Bridge Sampling Techniques

  2004influential reference
• Bayesian mixture model based clustering of replicated microarray data

  2004cited by this paper
• On the posterior distribution of the number of components in a finite mixture

  2004cited by this paper
• Bayesian clustering and product partition models

  2003influential reference
• Model-Based Clustering, Discriminant Analysis, and Density Estimation

  2002influential reference
• Modelling Heterogeneity With and Without the Dirichlet Process

  2001cited by this paper
• Finite Mixture Models

  2000cited by this paper
• MAXIMUM LIKELIHOOD AND BAYESIAN METHODS FOR MIXTURES OF NORMAL DISTRIBUTIONS

  1997cited by this paper
• The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator

  1997cited by this paper
• An entropy criterion for assessing the number of clusters in a mixture model

  1996cited by this paper
• Breast Cancer Diagnosis and Prognosis Via Linear Programming

  1995cited by this paper
• Bayesian Density Estimation and Inference Using Mixtures

  1995cited by this paper
• Estimation of Finite Mixture Distributions Through Bayesian Sampling

  1994influential reference
• An Introduction to Regression Graphics

  1994cited by this paper
• EXACT MEAN INTEGRATED SQUARED ERROR

  1992cited by this paper
• Bayesian nonparametric inference

  1992cited by this paper
• Expert system for predicting protein localization sites in gram‐negative bacteria

  1991influential reference
• A CONSTRUCTIVE DEFINITION OF DIRICHLET PRIORS

  1991cited by this paper
• BAYESIAN DENSITY ESTIMATION BY MIXTURES OF NORMAL DISTRIBUTIONS

  1983cited by this paper
• A k-means clustering algorithm

  1979cited by this paper
• Algorithm AS136: A k-means clustering algorithm.

  1979cited by this paper
• Contributions to the Theory of Dirichlet Processes

  1973cited by this paper
• Natural Scales in Geographical Patterns

  1971cited by this paper
• Some methods for classi cation and analysis of multivariate observations

  1967cited by this paper
• Some methods for classification and analysis of multivariate observations

  1967cited by this paper
• On the Use of Discriminant Functions in Taxonomy

  1962cited by this paper

• CliPS -- How to identify cluster distributions in Bayesian mixture models

  2026cites this paper
• pvEBayes: An R Package for Empirical Bayes Methods in Pharmacovigilance

  2025cites this paper
• BayesCurveFit: Enhancing Curve Fitting for Early-Stage Compound Screening Using Bayesian Inference

  2025cites this paper
• Flexible Empirical Bayesian Approaches to Pharmacovigilance for Simultaneous Signal Detection and Signal Strength Estimation in Spontaneous Reporting Systems Data

  2025cites this paper
• Bayesian mixture models with repulsive and attractive atoms

  2025cites this paper
• Mixture models for data with unknown distributions

  2025cites this paper
• HBMAP: Bayesian inference of neural circuits from DNA barcoded projection mapping

  2025cites this paper
• An Improved Copula‐Based Framework for Efficient Global Sensitivity Analysis

  2024cites this paper
• Hierarchical Mixture of Finite Mixtures (with Discussion)

  2024cites this paper
• Repulsion, chaos, and equilibrium in mixture models

  2024cites this paper
• Without pain – clustering categorical data using a Bayesian mixture of finite mixtures of latent class analysis models

  2024influential citation
• BAYESIAN LEARNING OF CLINICALLY MEANINGFUL SEPSIS PHENOTYPES IN NORTHERN TANZANIA.

  2024influential citation
• Estimating SARS-CoV-2 infection probabilities with serological data and a Bayesian mixture model

  2024cites this paper
• Automated detection of edge clusters via an overfitted mixture prior

  2024influential citation
• Advances in Bayesian random partition models: A comprehensive review

  2023cites this paper
• Coarsened Mixtures of Hierarchical Skew Normal Kernels for Flow and Mass Cytometry Analyses

  2023cites this paper
• Deep Mixture of Linear Mixed Models for Complex Longitudinal Data

  2023cites this paper
• Model-based clustering for network data via a latent shrinkage position cluster model

  2023cites this paper
• Refining COVID-19 retrospective diagnosis with continuous serological tests: a Bayesian mixture model

  2023cites this paper
• Some issues in robust clustering

  2023cites this paper
• Fast and order‐invariant inference in Bayesian VARs with nonparametric shocks

  2023cites this paper
• Bayesian Clustering via Fusing of Localized Densities

  2023cites this paper
• Bayesian cluster analysis

  2023cites this paper
• Missing data patterns in runners’ careers: do they matter?

  2022cites this paper
• pyVISCOUS: An open-source tool for computationally frugal global sensitivity analysis

  2022cites this paper
• A Semi-parametric Density Estimation with Application in Clustering

  2022cites this paper
• Risk analysis of stock investment using the value at risk methods with the Bayesian normal mixture approach

  2022cites this paper
• Spectral Clustering, Bayesian Spanning Forest, and Forest Process

  2022cites this paper
• Agglomerative and divisive hierarchical Bayesian clustering

  2022cites this paper
• Variance matrix priors for Dirichlet process mixture models with Gaussian kernels

  2022cites this paper
• Variational inference and sparsity in high-dimensional deep Gaussian mixture models

  2021influential citation
• intRinsic: An R Package for Model-Based Estimation of the Intrinsic Dimension of a Dataset

  2021cites this paper
• Margin-free classification and new class detection using finite Dirichlet mixtures

  2021cites this paper
• Review of Bayesian selection methods for categorical predictors using JAGS

  2021cites this paper
• Dynamic Dirichlet process mixture model for identifying voting coalitions in the United Nations General Assembly human rights roll call votes

  2021cites this paper
• Dynamic Mixture of Experts Models for Online Prediction

  2021cites this paper
• Strathclyde

  2021cites this paper
• Summarizing Finite Mixture Model with Overlapping Quantification

  2021cites this paper
• Dual-Supervised Uncertainty Inference of MoGMM-FC Layer for Image Recognition

  2021cites this paper
• Finite mixture models are typically inconsistent for the number of components

  2020cites this paper
• DS-UI: Dual-Supervised Mixture of Gaussian Mixture Models for Uncertainty Inference in Image Recognition

  2020cites this paper
• Bayesian mixture models for cytometry data analysis

  2020cites this paper
• Finite mixture models do not reliably learn the number of components

  2020cites this paper
• Generalized Mixtures of Finite Mixtures and Telescoping Sampling

  2020cites this paper
• Bayesian clustering of skewed and multimodal data using geometric skewed normal distributions

  2020cites this paper
• Better than the best? Answers via model ensemble in density-based clustering

  2020cites this paper
• An adequacy approach for deciding the number of clusters for OTRIMLE robust Gaussian mixture‐based clustering

  2020cites this paper
• Distributed Bayesian clustering using finite mixture of mixtures

  2020influential citation
• Dynamic mixtures of finite mixtures and telescoping sampling

  2020influential citation
• On the identifiability of Bayesian factor analytic models

  2020cites this paper
• Distributed Bayesian clustering

  2020cites this paper
• Enriched Pitman–Yor processes

  2020cites this paper
• Coarsened mixtures of hierarchical skew normal kernels for flow cytometry analyses

  2020cites this paper
• Discovery and Recognition of Emerging Human Activities Using a Hierarchical Mixture of Directional Statistical Models

  2020cites this paper
• An enriched mixture model for functional clustering

  2019cites this paper
• ePubWU Institutional Repository Sylvia Frühwirth-Schnatter and Gertraud Malsiner-Walli From here to infinity: sparse finite versus Dirichlet process mixtures in model- based clustering

  2019cites this paper
• How bettering the best? Answers via blending models and cluster formulations in density-based clustering

  2019cites this paper
• - Walli From here to infinity : sparse finite versus Dirichlet process mixtures in model-based clustering

  2019cites this paper
• Estimating densities with non‐linear support by using Fisher–Gaussian kernels

  2019cites this paper
• Bayesian nonparametric learning of how skill is distributed across the mutual fund industry

  2019cites this paper
• Clustering multivariate data using factor analytic Bayesian mixtures with an unknown number of components

  2019cites this paper
• ℓ0-Regularized Sparsity for Probabilistic Mixture Models

  2019cites this paper
• Is infinity that far? A Bayesian nonparametric perspective of finite mixture models

  2019cites this paper
• Hierarchical Normalized Completely Random Measures to Cluster Grouped Data

  2019cites this paper
• Semi-Supervised Non-Parametric Bayesian Modelling of Spatial Proteomics.

  2019cites this paper
• Scalable Bayesian Nonparametric Clustering and Classification

  2018cites this paper
• From here to infinity: sparse finite versus Dirichlet process mixtures in model-based clustering

  2018cites this paper
• Model-Based Clustering

  2018cites this paper
• Bayesian Distance Clustering

  2018cites this paper
• Model Selection for Mixture Models – Perspectives and Strategies

  2018influential citation
• The Sparse Latent Position Model for nonnegative weighted networks

  2018cites this paper
• From here to infinity - sparse finite versus Dirichlet process mixtures in model-based clustering

  2017cites this paper
• Introduction to Finite Mixtures

  2017cites this paper
• Effect fusion using model-based clustering

  2017cites this paper
• On choosing mixture components via non‐local priors

  2016cites this paper
• Bayesian Cluster Analysis: Point Estimation and Credible Balls (with Discussion)

  2015cites this paper