Generalizing Hamiltonian Monte Carlo with Neural Networks

We present a general-purpose method to train Markov chain Monte Carlo kernels, parameterized by deep neural networks, that converge and mix quickly to their target distribution. Our method generalizes Hamiltonian Monte Carlo and is trained to maximize expected squared jumped distance, a proxy for mixing speed. We demonstrate large empirical gains on a collection of simple but challenging distributions, for instance achieving a 106x improvement in effective sample size in one case, and mixing when standard HMC makes no measurable progress in a second. Finally, we show quantitative and qualitative gains on a real-world task: latent-variable generative modeling. We release an open source TensorFlow implementation of the algorithm.

• Publication year

  2017

• Venue

  International Conference on Learning Representations

• Publication date

  2017-11-25

• Fields of study

  Mathematics, Computer Science

• Identifiers
  arXiv 1711.09268
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• GENERATIVE ADVERSARIAL NETS

  2018cited by this paper
• Tackling Over-pruning in Variational Autoencoders

  2017cited by this paper
• A-NICE-MC: Adversarial Training for MCMC

  2017influential reference
• Can Boltzmann Machines Discover Cluster Updates ?

  2017cited by this paper
• Learning Deep Latent Gaussian Models with Markov Chain Monte Carlo

  2017cited by this paper
• Wasserstein GAN

  2017cited by this paper
• On the Quantitative Analysis of Decoder-Based Generative Models

  2016cited by this paper
• Magnetic Hamiltonian Monte Carlo

  2016cited by this paper
• Accelerated Monte Carlo simulations with restricted Boltzmann machines

  2016cited by this paper
• Self-learning Monte Carlo method

  2016cited by this paper
• Density estimation using Real NVP

  2016cited by this paper
• Deep Unsupervised Learning using Nonequilibrium Thermodynamics

  2015cited by this paper
• Exponential Integration for Hamiltonian Monte Carlo

  2015cited by this paper
• A note on the evaluation of generative models

  2015cited by this paper
• Importance Weighted Autoencoders

  2015cited by this paper
• Gradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families

  2015cited by this paper
• A Markov Jump Process for More Efficient Hamiltonian Monte Carlo

  2015cited by this paper
• Adam: A Method for Stochastic Optimization

  2014cited by this paper
• Hamiltonian Monte Carlo Without Detailed Balance

  2014influential reference
• Stochastic Gradient Hamiltonian Monte Carlo

  2014cited by this paper
• Markov Chain Monte Carlo and Variational Inference: Bridging the Gap

  2014cited by this paper
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models

  2014cited by this paper
• Auto-Encoding Variational Bayes

  2013cited by this paper
• Deep Generative Stochastic Networks Trainable by Backprop

  2013cited by this paper
• Violation of detailed balance accelerates relaxation.

  2013cited by this paper
• Kernel Adaptive Metropolis-Hastings

  2013cited by this paper
• Hamiltonian Annealed Importance Sampling for partition function estimation

  2012cited by this paper
• On the flexibility of the design of multiple try Metropolis schemes

  2012cited by this paper
• A General Metric for Riemannian Manifold Hamiltonian Monte Carlo

  2012cited by this paper
• MCMC Using Hamiltonian Dynamics

  2011influential reference
• Riemann manifold Langevin and Hamiltonian Monte Carlo methods

  2011cited by this paper
• The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo

  2011cited by this paper
• Probabilistic Inference Using Markov Chain Monte Carlo Methods

  2011cited by this paper
• Adaptive Rejection Sampling for Gibbs Sampling

  2010cited by this paper
• Riemannian Manifold Hamiltonian Monte Carlo

  2009cited by this paper
• A tutorial on adaptive MCMC

  2008cited by this paper
• Adaptively Scaling the Metropolis Algorithm Using Expected Squared Jumped Distance

  2007influential reference
• Simulating Hamiltonian dynamics

  2006cited by this paper
• The mnist database of handwritten digits

  2005cited by this paper
• Information Theory, Inference, and Learning Algorithms

  2004cited by this paper
• Shadow hybrid Monte Carlo: an efficient propagator in phase space of macromolecules

  2004cited by this paper
• An Introduction to MCMC for Machine Learning

  2004cited by this paper
• Geometric numerical integration illustrated by the Störmer–Verlet method

  2003cited by this paper
• An adaptive Metropolis algorithm

  2001cited by this paper
• A Direct Approach to Conformational Dynamics Based on Hybrid Monte Carlo

  1999cited by this paper
• Annealed importance sampling

  1998cited by this paper
• Sampling from multimodal distributions using tempered transitions

  1996cited by this paper
• Reversible jump Markov chain Monte Carlo computation and Bayesian model determination

  1995cited by this paper
• Two phase transitions in the fully frustrated XY model.

  1995cited by this paper
• Higher-order hybrid Monte Carlo algorithms.

  1989cited by this paper
• Hybrid Monte Carlo

  1988cited by this paper
• Mass tensor molecular dynamics

  1975cited by this paper
• Monte Carlo Sampling Methods Using Markov Chains and Their Applications

  1970influential reference
• Edinburgh Research Explorer Continuous Relaxations for Discrete Hamiltonian Monte Carlo

  year unknowncited by this paper

• Training neural control variates using correlated configurations

  2025cites this paper
• Neural Surrogate HMC: On Using Neural Likelihoods for Hamiltonian Monte Carlo in Simulation-Based Inference

  2024cites this paper
• Denoising Fisher Training For Neural Implicit Samplers

  2024cites this paper
• Hamiltonian Score Matching and Generative Flows

  2024cites this paper
• Training Neural Samplers with Reverse Diffusive KL Divergence

  2024cites this paper
• Neural Surrogate HMC: Accelerated Hamiltonian Monte Carlo with a Neural Network Surrogate Likelihood

  2024cites this paper
• Learning to Explore for Stochastic Gradient MCMC

  2024cites this paper
• Ai-Sampler: Adversarial Learning of Markov kernels with involutive maps

  2024influential citation
• S2AC: Energy-Based Reinforcement Learning with Stein Soft Actor Critic

  2024cites this paper
• Sampling thermodynamic ensembles of molecular systems with generative neural networks: Will integrating physics-based models close the generalization gap?

  2024cites this paper
• Repelling-Attracting Hamiltonian Monte Carlo

  2024cites this paper
• Energy based diffusion generator for efficient sampling of Boltzmann distributions

  2024cites this paper
• Reliability Analysis of Complex Systems using Subset Simulations with Hamiltonian Neural Networks

  2024cites this paper
• Timewarp: Transferable Acceleration of Molecular Dynamics by Learning Time-Coarsened Dynamics

  2023cites this paper
• MLMC: Machine Learning Monte Carlo for Lattice Gauge Theory

  2023cites this paper
• Data-Efficient Generation of Protein Conformational Ensembles with Backbone-to-Side-Chain Transformers.

  2023cites this paper
• Energy-Guided Continuous Entropic Barycenter Estimation for General Costs

  2023cites this paper
• Self-Tuning Hamiltonian Monte Carlo for Accelerated Sampling

  2023cites this paper
• Learning variational autoencoders via MCMC speed measures

  2023cites this paper
• CSE 272 Assignment 3 : Final Project

  2023cites this paper
• Entropy-based Training Methods for Scalable Neural Implicit Sampler

  2023cites this paper
• Optimal Preconditioning and Fisher Adaptive Langevin Sampling

  2023cites this paper
• Machine Learning and the Future of Bayesian Computation

  2023influential citation
• A Diffusion-Based Method for Multi-Turn Compositional Image Generation

  2023cites this paper
• Adaptive weighting of Bayesian physics informed neural networks for multitask and multiscale forward and inverse problems

  2023cites this paper
• Zero-shot-Learning Cross-Modality Data Translation Through Mutual Information Guided Stochastic Diffusion

  2023cites this paper
• Quantification of Predictive Uncertainty via Inference-Time Sampling

  2023cites this paper
• Applications of Machine Learning to Lattice Quantum Field Theory

  2022cites this paper
• Practical tradeoffs between memory, compute, and performance in learned optimizers

  2022cites this paper
• Toward Unlimited Self-Learning MCMC with Parallel Adaptive Annealing

  2022cites this paper
• VeLO: Training Versatile Learned Optimizers by Scaling Up

  2022cites this paper
• Aspects of scaling and scalability for flow-based sampling of lattice QCD

  2022cites this paper
• Probabilistic Safety Assessment in Composite Materials using BNN by ABC-SS

  2022cites this paper
• Toward Unlimited Self-Learning Monte Carlo with Annealing Process Using VAE's Implicit Isometricity

  2022cites this paper
• Denoising MCMC for Accelerating Diffusion-Based Generative Models

  2022cites this paper
• Adversarially Training MCMC with Non-Volume-Preserving Flows

  2022influential citation
• Parameterized Hamiltonian Learning With Quantum Circuit

  2022cites this paper
• Bayesian Inference with Latent Hamiltonian Neural Networks

  2022cites this paper
• Flow Annealed Importance Sampling Bootstrap

  2022cites this paper
• Enhanced gradient-based MCMC in discrete spaces

  2022cites this paper
• Fixed-Distance Hamiltonian Monte Carlo

  2022influential citation
• Accelerating Hamiltonian Monte Carlo via Chebyshev Integration Time

  2022cites this paper
• Parallel Tempering With a Variational Reference

  2022influential citation
• Arbitrary conditional inference in variational autoencoders via fast prior network training

  2022cites this paper
• Artificial Intelligence and Machine Learning in Nuclear Physics

  2021cites this paper
• Nested sampling with normalizing flows for gravitational-wave inference

  2021cites this paper
• Decentralized Langevin Dynamics over a Directed Graph

  2021cites this paper
• Improving Actor-Critic Reinforcement Learning via Hamiltonian Policy

  2021cites this paper
• Antithetic Magnetic and Shadow Hamiltonian Monte Carlo

  2021cites this paper
• Revisiting Bayesian autoencoders with MCMC

  2021cites this paper
• Non-Volume Preserving Hamiltonian Monte Carlo and No-U-TurnSamplers

  2021influential citation
• Bayesian Graph Convolutional Neural Networks via Tempered MCMC

  2021influential citation
• Deep Learning Hamiltonian Monte Carlo

  2021influential citation
• Finding the deconfinement temperature in lattice Yang-Mills theories from outside the scaling window with machine learning

  2021cites this paper
• Unbiased Monte Carlo Cluster Updates with Autoregressive Neural Networks

  2021cites this paper
• Adaptive Monte Carlo augmented with normalizing flows

  2021influential citation
• DLIO: A Data-Centric Benchmark for Scientific Deep Learning Applications

  2021cites this paper
• Semi-Empirical Objective Functions for MCMC Proposal Optimization

  2021cites this paper
• Reparameterized Sampling for Generative Adversarial Networks

  2021influential citation
• Solution of Physics-based Bayesian Inverse Problems with Deep Generative Priors

  2021influential citation
• A Gradient Based Strategy for Hamiltonian Monte Carlo Hyperparameter Optimization

  2021cites this paper
• Efficient Bayesian Sampling Using Normalizing Flows to Assist Markov Chain Monte Carlo Methods

  2021cites this paper
• Structured Stochastic Gradient MCMC

  2021cites this paper
• NEO: Non Equilibrium Sampling on the Orbit of a Deterministic Transform

  2021cites this paper
• LSB: Local Self-Balancing MCMC in Discrete Spaces

  2021cites this paper
• S EMI -E MPIRICAL O BJECTIVE F UNCTIONS FOR N EURAL MCMC P ROPOSAL O PTIMIZATION

  2021influential citation
• Delayed rejection Hamiltonian Monte Carlo for sampling multiscale distributions

  2021cites this paper
• Quantum-Inspired Magnetic Hamiltonian Monte Carlo

  2021cites this paper
• Entropy-based adaptive Hamiltonian Monte Carlo

  2021cites this paper
• Hamiltonian Dynamics with Non-Newtonian Momentum for Rapid Sampling

  2021cites this paper
• HAAR-WEAVE-METROPOLIS KERNEL

  2021cites this paper
• Bootstrap Your Flow

  2021influential citation
• Locally Scaled and Stochastic Volatility Metropolis- Hastings Algorithms

  2021cites this paper
• LeapfrogLayers: A Trainable Framework for Effective Topological Sampling

  2021cites this paper
• Machine Learning in Nuclear Physics

  2021cites this paper
• Involutive MCMC: a Unifying Framework

  2020cites this paper
• Machine learning for

  2020cites this paper
• On the accept-reject mechanism for Metropolis-Hastings algorithms

  2020cites this paper
• A Neural Network MCMC Sampler That Maximizes Proposal Entropy

  2020influential citation
• Denoising Diffusion Implicit Models

  2020cites this paper
• Machine-learning physics from unphysics: Finding deconfinement temperature in lattice Yang-Mills theories from outside the scaling window

  2020cites this paper
• Acceleration of Structural Analysis Simulations using CNN-based Auto-Tuning of Solver Tolerance

  2020cites this paper
• Deep Involutive Generative Models for Neural MCMC

  2020cites this paper
• Denoising Diffusion Probabilistic Models

  2020cites this paper
• Understanding and mitigating exploding inverses in invertible neural networks

  2020cites this paper
• Joint Stochastic Approximation and Its Application to Learning Discrete Latent Variable Models

  2020cites this paper
• MetFlow: A New Efficient Method for Bridging the Gap between Markov Chain Monte Carlo and Variational Inference

  2020influential citation
• Stochastic Normalizing Flows

  2020cites this paper
• Augmented Normalizing Flows: Bridging the Gap Between Generative Flows and Latent Variable Models

  2020cites this paper
• i- flow: High-dimensional integration and sampling with normalizing flows

  2020cites this paper
• Machine Learning and Neural Networks for Field Theory

  2020influential citation
• From Interatomic Distances to Protein Tertiary Structures with a Deep Convolutional Neural Network

  2020cites this paper
• Understanding I/O behavior of Scientific Deep Learning Applications in HPC systems

  2020cites this paper
• Generative deep learning for macromolecular structure and dynamics.

  2020cites this paper
• Nonreversible MCMC from conditional invertible transforms: a complete recipe with convergence guarantees

  2020cites this paper
• $(1 + \varepsilon)$-class Classification: an Anomaly Detection Method for Highly Imbalanced or Incomplete Data Sets

  2019cites this paper
• ETA-L EARNING FOR S TOCHASTIC G RADIENT MCMC

  2019cites this paper
• NeuTra-lizing Bad Geometry in Hamiltonian Monte Carlo Using Neural Transport

  2019cites this paper
• Sequence network 1 D 2 D Structure network Invariant features A C OrientationsDistances B weights features Energy

  2019cites this paper
• Robust Biophysical Parameter Estimation with a Neural Network Enhanced Hamiltonian Markov Chain Monte Carlo Sampler

  2019influential citation