Statistical and Computational Tradeoffs in Stochastic Composite Likelihood

Maximum likelihood estimators are often of limited practical use due to the intensive computation they require. We propose a family of alternative estimators that maximize a stochastic variation of the composite likelihood function. We prove the consistency of the estimators, provide formulas for their asymptotic variance and computational complexity, and discuss experimental results in the context of Boltzmann machines and conditional random fields. The theoretical and experimental studies demonstrate the effectiveness of the estimators in achieving a predefined balance between computational complexity and statistical accuracy.

• Publication year

  2009

• Venue

  International Conference on Artificial Intelligence and Statistics

• Publication date

  2009-04-15

• Fields of study

  Mathematics, Computer Science

• Identifiers
  arXiv 1003.0691
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Non-Local Contrastive Objectives

  2010cited by this paper
• An asymptotic analysis of generative, discriminative, and pseudolikelihood estimators

  2008cited by this paper
• Piecewise pseudolikelihood for efficient training of conditional random fields

  2007cited by this paper
• Isotonic Conditional Random Fields and Local Sentiment Flow

  2006cited by this paper
• Elements of Information Theory

  2005cited by this paper
• A note on composite likelihood inference and model selection

  2005cited by this paper
• Monte Carlo Statistical Methods

  2004cited by this paper
• RCV1: A New Benchmark Collection for Text Categorization Research

  2004cited by this paper
• Maximum pseudo likelihood estimation in network tomography

  2003cited by this paper
• Shallow Parsing with Conditional Random Fields

  2003cited by this paper
• A generalized mean field algorithm for variational inference in exponential families

  2002cited by this paper
• Learning in Gibbsian fields: how accurate and how fast can it be?

  2000cited by this paper
• An Introduction to Variational Methods for Graphical Models

  1999influential reference
• Equivalence of Linear Boltzmann Chains and Hidden Markov Models

  1996cited by this paper
• A Course in Large Sample Theory

  1996influential reference
• Matrix analysis

  1985cited by this paper
• OPTIMAL PERCEPTUAL INFERENCE

  1983cited by this paper
• Approximation Theorems of Mathematical Statistics

  1980cited by this paper
• Discrete Multivariate Analysis: Theory and Practice

  1976cited by this paper
• Spatial Interaction and the Statistical Analysis of Lattice Systems

  1974cited by this paper

• Statistical and Computational Trade-offs in Variational Inference: A Case Study in Inferential Model Selection

  2022cites this paper
• Second Order Pseudolikelihood Learning in Relational Domain

  2019cites this paper
• Bottom-Up Approaches to Approximate Inference and Learning in Discrete Graphical Models

  2014cites this paper
• Continuous Graphical Models for Static and Dynamic Distributions: Application to Structural Biology

  2013cites this paper
• Constructing Composite Likelihoods in General Random Fields

  2013cites this paper
• Distributed Covariance Estimation in

  2012cites this paper
• Continuous Graphical Models for Static and Dynamic Distributions : Application to Structural Biology

  2012cites this paper
• The von Mises Graphical Model: Structure Learning (CMU-CS-11-108 / CMU-CB-11-100)

  2011cites this paper
• The von Mises Graphical Model: Regularized Structure and Parameter Learning (CMU-CS-11-129/CMU-CB-11-101)

  2011cites this paper
• Relational Learning with One Network: An Asymptotic Analysis

  2011cites this paper
• The von Mises Graphical Model : Structure Learning

  2011cites this paper
• ISSUES AND STRATEGIES IN THE SELECTION OF COMPOSITE LIKELIHOODS

  2011influential citation
• Asymptotic Analysis of Generative Semi-Supervised Learning

  2010cites this paper
• Distributed Covariance Estimation in Gaussian Graphical Models

  2010cites this paper
• Learning with Blocks: Composite Likelihood and Contrastive Divergence

  2010cites this paper
• An Introduction to Conditional Random Fields

  2010cites this paper
• Learning without labels and nonnegative tensor factorization

  2010cites this paper