Stable Estimation of a Covariance Matrix Guided by Nuclear Norm Penalties

Estimation of a covariance matrix or its inverse plays a central role in many statistical methods. For these methods to work reliably, estimated matrices must not only be invertible but also well-conditioned. The current paper introduces a novel prior to ensure a well-conditioned maximum a posteriori (MAP) covariance estimate. The prior shrinks the sample covariance estimator towards a stable target and leads to a MAP estimator that is consistent and asymptotically efficient. Thus, the MAP estimator gracefully transitions towards the sample covariance matrix as the number of samples grows relative to the number of covariates. The utility of the MAP estimator is demonstrated in two standard applications - discriminant analysis and EM clustering - in this sampling regime.

• Publication year

  2013

• Venue

  Computational Statistics & Data Analysis

• Publication date

  2013-05-14

• Fields of study

  Mathematics, Computer Science, Medicine

• Identifiers
  DOI 10.1016/j.csda.2014.06.018arXiv 1305.3312PMID 25143662PMCID PMC4133137
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• FINDING STRUCTURE WITH RANDOMNESS : PROBABILISTIC ALGORITHMS FOR CONSTRUCTING

  2016cited by this paper
• Condition‐number‐regularized covariance estimation

  2013cited by this paper
• Identification of fever and vaccine-associated gene interaction networks using ontology-based literature mining

  2012cited by this paper
• Hub Discovery in Partial Correlation Graphs

  2012cited by this paper
• MINIMAX ESTIMATION OF LARGE COVARIANCE MATRICES UNDER ℓ1-NORM

  2012cited by this paper
• Nonlinear Shrinkage Estimation of Large-Dimensional Covariance Matrices

  2012cited by this paper
• Large-Scale Correlation Screening

  2011cited by this paper
• Covariance Estimation: The GLM and Regularization Perspectives

  2011cited by this paper
• HIGH DIMENSIONAL COVARIANCE MATRIX ESTIMATION IN APPROXIMATE FACTOR MODELS.

  2011cited by this paper
• Regularized Parameter Estimation in High-Dimensional Gaussian Mixture Models

  2011cited by this paper
• Weighted Network Analysis

  2011cited by this paper
• Wishart distributions for decomposable covariance graph models

  2011cited by this paper
• Estimation of high-dimensional low-rank matrices

  2009cited by this paper
• Estimation of Covariance Matrix

  2009cited by this paper
• Covariance‐regularized regression and classification for high dimensional problems

  2009influential reference
• Partial Correlation Estimation by Joint Sparse Regression Models

  2008cited by this paper
• Finite Mixture Models

  2008cited by this paper
• FLEXIBLE COVARIANCE ESTIMATION IN GRAPHICAL GAUSSIAN MODELS

  2008cited by this paper
• Sparse inverse covariance estimation with the graphical lasso.

  2008cited by this paper
• CCA: An R Package to Extend Canonical Correlation Analysis

  2008cited by this paper
• Regularized estimation of large covariance matrices

  2008cited by this paper
• Penalized Normal Likelihood and Ridge Regularization of Correlation and Covariance Matrices

  2008cited by this paper
• High-dimensional covariance estimation by minimizing ℓ1-penalized log-determinant divergence

  2008cited by this paper
• k-means++: the advantages of careful seeding

  2007cited by this paper
• Bayesian Regularization for Normal Mixture Estimation and Model-Based Clustering

  2007cited by this paper
• Simultaneous modelling of the Cholesky decomposition of several covariance matrices

  2007cited by this paper
• Novel aspects of PPARα‐mediated regulation of lipid and xenobiotic metabolism revealed through a nutrigenomic study

  2007cited by this paper
• Constrained monotone EM algorithms for finite mixture of multivariate Gaussians

  2007cited by this paper
• Regularized linear discriminant analysis and its application in microarrays.

  2007cited by this paper
• Covariance matrix selection and estimation via penalised normal likelihood

  2006cited by this paper
• Addendum: Regularization and variable selection via the elastic net

  2005cited by this paper
• Book Review: Mathematical and statistical methods for genetic analysis, 2nd edition

  2005cited by this paper
• A likelihood-based constrained algorithm for multivariate normal mixture models

  2004cited by this paper
• A well-conditioned estimator for large-dimensional covariance matrices

  2004influential reference
• Estimation of the multivariate normal covariance matrix under some restrictions

  2003cited by this paper
• Nonparametric estimation of large covariance matrices of longitudinal data

  2003cited by this paper
• Shrinkage Estimators for Covariance Matrices

  2001cited by this paper
• Ridge Regression: Biased Estimation for Nonorthogonal Problems

  2000cited by this paper
• Multivariate Analysis

  1998influential reference
• A Course in Large Sample Theory

  1996influential reference
• Regression Shrinkage and Selection via the Lasso

  1996cited by this paper
• Canonical correlation analysis when the data are curves.

  1993cited by this paper
• The Variational Form of Certain Bayes Estimators

  1991cited by this paper
• Regularized Discriminant Analysis

  1989cited by this paper
• Estimation of a covariance matrix under Stein's loss

  1985cited by this paper
• A Constrained Formulation of Maximum-Likelihood Estimation for Normal Mixture Distributions

  1985cited by this paper
• Canonical ridge and econometrics of joint production

  1976cited by this paper
• A trace inequality of John von Neumann

  1975cited by this paper
• DISTRIBUTION OF EIGENVALUES FOR SOME SETS OF RANDOM MATRICES

  1967cited by this paper
• Relations Between Two Sets of Variates

  1936cited by this paper
• Stat260/cs294: Randomized Algorithms for Matrices and Data

  year unknowncited by this paper
• Model Selection Through Sparse Max Likelihood Estimation Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data

  year unknowncited by this paper

• Rethinking Structural Equation Modeling in Political Science: Challenges, Best Practices, and Future Directions

  2025cites this paper
• A unified approach to penalized likelihood estimation of covariance matrices in high dimensions

  2024cites this paper
• Improved methods for empirical Bayes multivariate multiple testing and effect size estimation

  2024cites this paper
• Learning Covariances for Estimation with Constrained Bilevel Optimization

  2023cites this paper
• Decoding Attended Spatial Location during Complex Scene Analysis with fNIRS

  2022cites this paper
• Evaluation of Goodness-of-Fit Tests in Random Intercept Cross-Lagged Panel Model: Implications for Small Samples

  2022cites this paper
• RGLS and RLS in Covariance Structure Analysis

  2021influential citation
• A proximal distance algorithm for likelihood-based sparse covariance estimation.

  2021cites this paper
• Testing Mean and Covariance Structures with Reweighted Least Squares

  2021cites this paper
• Sparse sketches with small inversion bias

  2020cites this paper
• MDiNE: a model to estimate differential co-occurrence networks in microbiome studies

  2019cites this paper
• Bibliography

  2019cites this paper
• Model-based Classification

  2019cites this paper
• Network data

  2019cites this paper
• The spectral condition number plot for regularization parameter evaluation

  2019cites this paper
• Model-Based Clustering and Classification for Data Science: With Applications in R

  2019cites this paper
• High-dimensional Data

  2019cites this paper
• Other Topics

  2019cites this paper
• Model-Based Clustering and Classification for Data Science

  2019cites this paper
• Variable Selection

  2019cites this paper
• Dealing with Difficulties

  2019cites this paper
• Discrete Data Clustering

  2019cites this paper
• Preface

  2019cites this paper
• Non-Gaussian Model-based Clustering

  2019cites this paper
• Model-based Clustering with Covariates

  2019cites this paper
• Model-based Clustering: Basic Ideas

  2019cites this paper
• Semi-supervised Clustering and Classification

  2019cites this paper
• Introduction

  2019cites this paper
• Estimating Large Correlation Matrices for International Migration April

  2018influential citation
• An overview of large‐dimensional covariance and precision matrix estimators with applications in chemometrics

  2017cites this paper
• A Regularized GLS for Structural Equation Modeling

  2017influential citation
• Estimating Large Correlation Matrices for International Migration.

  2016influential citation
• Projection and Estimation of International Migration

  2016cites this paper
• Human immunophenotyping via low-variance, low-bias, interpretive regression modeling of small, wide data sets: Application to aging and immune response to influenza vaccination.

  2016cites this paper
• Comparative Analysis of Covariance Matrix Estimation for Anomaly Detection in Hyperspectral Images

  2015cites this paper
• On the Solution Path of Regularized Covariance Estimators

  2015cites this paper
• Subject Index

  1984cites this paper