High-dimensional Covariance Estimation Based On Gaussian Graphical Models

Undirected graphs are often used to describe high dimensional distributions. Under sparsity conditions, the graph can be estimated using l1-penalization methods. We propose and study the following method. We combine a multiple regression approach with ideas of thresholding and refitting: first we infer a sparse undirected graphical model structure via thresholding of each among many l1-norm penalized regression functions; we then estimate the covariance matrix and its inverse using the maximum likelihood estimator. We show that under suitable conditions, this approach yields consistent estimation in terms of graphical structure and fast convergence rates with respect to the operator and Frobenius norm for the covariance matrix and its inverse. We also derive an explicit bound for the Kullback Leibler divergence.

• Publication year

  2010

• Venue

  Journal of machine learning research

• Publication date

  2010-09-02

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.5555/1953048.2078201arXiv 1009.0530
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• The adaptive and the thresholded Lasso for potentially misspecified models (and a lower bound for the Lasso)

  2011cited by this paper
• A Constrained ℓ1 Minimization Approach to Sparse Precision Matrix Estimation

  2011cited by this paper
• Regularization Paths for Generalized Linear Models via Coordinate Descent.

  2010cited by this paper
• High Dimensional Inverse Covariance Matrix Estimation via Linear Programming

  2010cited by this paper
• Adaptive estimation of covariance matrices via Cholesky decomposition

  2010cited by this paper
• Geometry of maximum likelihood estimation in Gaussian graphical models

  2010cited by this paper
• Thresholding Procedures for High Dimensional Variable Selection and Statistical Estimation

  2009influential reference
• Restricted Eigenvalue Conditions on Subgaussian Random Matrices

  2009cited by this paper
• Restricted Isometry Property of Matrices with Independent Columns and Neighborly Polytopes by Random Sampling

  2009cited by this paper
• Adaptive Lasso for High Dimensional Regression and Gaussian Graphical Modeling

  2009cited by this paper
• NETWORK EXPLORATION VIA THE ADAPTIVE LASSO AND SCAD PENALTIES.

  2009cited by this paper
• High dimensional sparse covariance estimation via directed acyclic graphs

  2009cited by this paper
• LASSO-TYPE RECOVERY OF SPARSE REPRESENTATIONS FOR HIGH-DIMENSIONAL DATA

  2008cited by this paper
• Time varying undirected graphs

  2008cited by this paper
• High-dimensional covariance estimation by minimizing ℓ1-penalized log-determinant divergence

  2008cited by this paper
• A note on the Lasso for Gaussian graphical model selection

  2008cited by this paper
• Regularized estimation of large covariance matrices

  2008cited by this paper
• SIMULTANEOUS ANALYSIS OF LASSO AND DANTZIG SELECTOR

  2008cited by this paper
• One-step Sparse Estimates in Nonconcave Penalized Likelihood Models.

  2008cited by this paper
• Sparse estimation of large covariance matrices via a nested Lasso penalty

  2008cited by this paper
• Discussion: One-step sparse estimates in nonconcave penalized likelihood models

  2008cited by this paper
• Discussion: One-step sparse estimates in nonconcave penalized likelihood models

  2008cited by this paper
• Sparse inverse covariance estimation with the graphical lasso.

  2008cited by this paper
• Partial Correlation Estimation by Joint Sparse Regression Models

  2008influential reference
• Sparsistency and Rates of Convergence in Large Covariance Matrices Estimation

  2007influential reference
• Sparsistency and Rates of Convergence in Large Covariance Matrix Estimation.

  2007influential reference
• Model selection and estimation in the Gaussian graphical model

  2007influential reference
• Estimation of high-dimensional prior and posterior covariance matrices in Kalman filter variants

  2007cited by this paper
• Relaxed Lasso

  2007cited by this paper
• Covariance matrix selection and estimation via penalised normal likelihood

  2006cited by this paper
• On Model Selection Consistency of Lasso

  2006cited by this paper
• High-dimensional graphs and variable selection with the Lasso

  2006cited by this paper
• The Adaptive Lasso and Its Oracle Properties

  2006influential reference
• First-Order Methods for Sparse Covariance Selection

  2006cited by this paper
• Estimation of a covariance matrix with zeros

  2005cited by this paper
• The Dantzig selector: Statistical estimation when P is much larger than n

  2005cited by this paper
• Some theory for Fisher''s linear discriminant function

  2004cited by this paper
• Sparse graphical Gaussian modeling of the isoprenoid gene network in Arabidopsis thaliana

  2004cited by this paper
• Nonparametric estimation of large covariance matrices of longitudinal data

  2003cited by this paper
• Predicting the clinical status of human breast cancer by using gene expression profiles

  2001cited by this paper
• Chi-square oracle inequalities

  2000cited by this paper
• Model Selection Through Sparse Max Likelihood Estimation Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data

  year unknowncited by this paper
• Covariance Estimation

  year unknowncited by this paper
• 25th Annual Conference on Learning Theory Reconstruction from Anisotropic Random Measurements

  year unknowncited by this paper

• Detection of degree distribution for biological networks in pearson family and its approximation

  2025cites this paper
• Gaussian Graphical Model Selection Using Graph Compression

  2025cites this paper
• Cluster Gelnet for estimating Gaussian graphical models with multi-level conditional correlations and block structures

  2025cites this paper
• A new hierarchical distribution on arbitrary sparse precision matrices

  2025cites this paper
• A network analysis of affective and motivational individual differences and error monitoring in a non-clinical sample

  2024cites this paper
• Graph Laplace Regularization-based pressure sensor placement strategy for leak localization in the water distribution networks under joint hydraulic and topological feature spaces.

  2024cites this paper
• Large Precision Matrix Estimation with Unknown Group Structure

  2024cites this paper
• Joint estimation for multisource Gaussian graphical models based on transfer learning

  2024cites this paper
• Sparse Quadratic Approximation for Graph Learning

  2023cites this paper
• Consistent causal inference for high-dimensional time series

  2023cites this paper
• Estimation of Graphical Models: An Overview of Selected Topics

  2023cites this paper
• HES: Edge Sampling for Heterogeneous Graphs

  2023cites this paper
• Information criteria for structured parameter selection in high dimensional tree and graph models

  2023cites this paper
• ARCH: Large-scale Knowledge Graph via Aggregated Narrative Codified Health Records Analysis

  2023cites this paper
• Finite sample rates of convergence for the Bigraphical and Tensor graphical Lasso estimators

  2023cites this paper
• A Unified Framework for Change Point Detection in High-Dimensional Linear Models

  2022cites this paper
• Bayesian Learning of Graph Substructures

  2022cites this paper
• Testing the Graph of a Gaussian Graphical Model

  2022cites this paper
• GraphSPME: Markov Precision Matrix Estimation and Asymptotic Stein-Type Shrinkage

  2022cites this paper
• Network Modeling in Biology: Statistical Methods for Gene and Brain Networks.

  2021cites this paper
• A 2-stage elastic net algorithm for estimation of sparse networks with heavy-tailed data

  2021cites this paper
• The G-Wishart Weighted Proposal Algorithm: Efficient Posterior Computation for Gaussian Graphical Models

  2021cites this paper
• High‐dimensional precision matrix estimation with a known graphical structure

  2021cites this paper
• Ensemble sparse estimation of covariance structure for exploring genetic disease data

  2021cites this paper
• Graphical Elastic Net and Target Matrices: Fast Algorithms and Software for Sparse Precision Matrix Estimation

  2021cites this paper
• Learning Gaussian Graphical Models via Multiplicative Weights

  2020cites this paper
• Effective Learning of a GMRF Mixture Model

  2020cites this paper
• Gaussian Graphical Model Exploration and Selection in High Dimension Low Sample Size Setting

  2020cites this paper
• A High-Dimensional Choice Model for Online Retailing

  2020cites this paper
• The Sylvester Graphical Lasso (SyGlasso)

  2020cites this paper
• Optimized Elastic Network Models With Direct Characterization of Inter-Residue Cooperativity for Protein Dynamics

  2020influential citation
• A procedure of linear discrimination analysis with detected sparsity structure for high-dimensional multi-class classification

  2020cites this paper
• A two-step method for estimating high-dimensional Gaussian graphical models

  2020cites this paper
• Reconstruction of Bio-molecular Networks

  2020cites this paper
• Markov Random Fields for Collaborative Filtering

  2019cites this paper
• Markov Random Fields for Collaborative Filtering

  2019cites this paper
• Network Inference From Consensus Dynamics With Unknown Parameters

  2019cites this paper
• Inference under Fine-Gray competing risks model with high-dimensional covariates

  2019cites this paper
• Gextext: Disease Network Extraction from Biomedical Literature.

  2019cites this paper
• A Stepwise Approach for High-Dimensional Gaussian Graphical Models

  2018cites this paper
• Graph Learning From Filtered Signals: Graph System and Diffusion Kernel Identification

  2018cites this paper
• Multiple matrix Gaussian graphs estimation

  2018cites this paper
• Modeling of Biochemical Networks via Classification and Regression Tree Methods

  2018cites this paper
• Edge selection for undirected graphs

  2018cites this paper
• High Dimensional Covariance Estimation for Spatio-Temporal Processes

  2017cites this paper
• Robust and sparse estimation of high-dimensional precision matrices via bivariate outlier detection

  2017cites this paper
• Neighborhood selection with application to social networks

  2017influential citation
• Fine-Gray competing risks model with high-dimensional covariates: estimation and Inference

  2017cites this paper
• Time-dependent spatially varying graphical models, with application to brain fMRI data analysis

  2017cites this paper
• Model-based clustering with sparse covariance matrices

  2017cites this paper
• MARS as an alternative approach of Gaussian graphical model for biochemical networks

  2017cites this paper
• Tensor graphical lasso (TeraLasso)

  2017cites this paper
• Graph Learning From Data Under Laplacian and Structural Constraints

  2016cites this paper
• Joint Structural Estimation of Multiple Graphical Models

  2016cites this paper
• Graph Learning from Data under Structural and Laplacian Constraints

  2016cites this paper
• Estimating covariance structure in high dimensions

  2016cites this paper
• Joint Mean and Covariance Estimation with Unreplicated Matrix-Variate Data

  2016cites this paper
• Selected Problems for High-Dimensional Data -- Quantile and Errors-in-Variables Regressions

  2016cites this paper
• Compressible Reparametrization of Time-Variant Linear Dynamical Systems

  2016cites this paper
• On Gaussian Markov models for conditional independence

  2016cites this paper
• A review of undirected and acyclic directed Gaussian Markov model selection and estimation

  2016cites this paper
• Efficient estimation of approximate factor models via penalized maximum likelihood

  2016cites this paper
• Monotone regression in high (and lower) dimensions

  2016cites this paper
• Estimation and Inference for High-Dimensional Gaussian Graphical Models with Structural Constraints.

  2015cites this paper
• Problems in Network Modeling: Estimating Edges and Community Detection

  2015cites this paper
• Mixed Effects Models for Resampled Network Statistics Improves Statistical Power to Find Differences in Multi-Subject Functional Connectivity

  2015cites this paper
• Gene Network Reconstruction using Global-Local Shrinkage Priors.

  2015cites this paper
• On model misspecification and KL separation for Gaussian graphical models

  2015cites this paper
• Towards Bicycle Demand Prediction of Large-Scale Bicycle Sharing System

  2014cites this paper
• Knowledge-fused Identification of Condition-specific Rewiring of Dependencies in Biological Networks

  2014cites this paper
• A joint convex penalty for inverse covariance matrix estimation

  2014cites this paper
• Partial correlation matrix estimation using ridge penalty followed by thresholding and re‐estimation

  2014cites this paper
• High Dimensional Separable Representations for Statistical Estimation and Controlled Sensing

  2014cites this paper
• Contributions to Penalized Estimation

  2014cites this paper
• Inferring gene-gene interactions and functional modules using sparse canonical correlation analysis

  2014cites this paper
• Estimation of positive definite M-matrices and structure learning for attractive Gaussian Markov Random fields

  2014influential citation
• A Well-Conditioned and Sparse Estimation of Covariance and Inverse Covariance Matrices Using a Joint Penalty

  2014cites this paper
• Review on statistical methods for gene network reconstruction using expression data.

  2014cites this paper
• Network-based pathway enrichment analysis with incomplete network information

  2014cites this paper
• Penalized Covariance Matrix Estimation Using a Matrix-Logarithm Transformation

  2013cites this paper
• Inverse Covariance Estimation for High-Dimensional Data in Linear Time and Space: Spectral Methods for Riccati and Sparse Models

  2013cites this paper
• Graphical models for high dimensional genomic data

  2013cites this paper
• RANK-BASED ESTIMATION OF NONPARANORMAL GRAPHICAL MODELS 2543 TABLE 1 Testing for normality of the gene expression measurements in the Arabidposis thaliana data

  2013cites this paper
• Canonical correlation analysis for RNA-seq co-expression networks

  2013cites this paper
• Gemini: Graph estimation with matrix variate normal instances

  2012cites this paper
• Efficient Estimation of Approximate Factor Models

  2012cites this paper
• Regularized Learning of High-dimensional Sparse Graphical Models

  2012cites this paper
• Regularized rank-based estimation of high-dimensional nonparanormal graphical models

  2012cites this paper
• On the Statistical Efficiency of $\ell_{1,p}$ Multi-Task Learning of Gaussian Graphical Models

  2012cites this paper
• High-dimensional estimation using graphical models and clustering

  2012cites this paper
• Exact Covariance Thresholding into Connected Components for Large-Scale Graphical Lasso

  2011cites this paper
• Thresholded Lasso for high dimensional variable selection

  2010influential citation
• ISCOVERY AND E XPANSION OF N EW D OMAINS WITHIN D IFFUSION

  year unknowncites this paper