General state space Markov chains and MCMC algorithms

This paper surveys various results about Markov chains on gen- eral (non-countable) state spaces. It begins with an introduction to Markov chain Monte Carlo (MCMC) algorithms, which provide the motivation and context for the theory which follows. Then, sucient conditions for geomet- ric and uniform ergodicity are presented, along with quantitative bounds on the rate of convergence to stationarity. Many of these results are proved using direct coupling constructions based on minorisation and drift con- ditions. Necessary and sucient conditions for Central Limit Theorems (CLTs) are also presented, in some cases proved via the Poisson Equa- tion or direct regeneration constructions. Finally, optimal scaling and weak convergence results for Metropolis-Hastings algorithms are discussed. None of the results presented is new, though many of the proofs are. We also describe some Open Problems.

• Publication year

  2004

• Venue

  Probability Surveys

• Publication date

  2004-04-02

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1214/154957804100000024arXiv math/0404033
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Subgeometric rates of convergence of f-ergodic Markov chains

  2006cited by this paper
• Scaling limits for the transient phase of local Metropolis–Hastings algorithms

  2005cited by this paper
• On adaptive Markov chain Monte Carlo algorithms

  2005cited by this paper
• Renewal theory and computable convergence rates for geometrically ergodic Markov chains

  2005cited by this paper
• On the central limit theorem for geometrically ergodic Markov chains

  2005cited by this paper
• Markov Processes: Characterization and Convergence

  2005cited by this paper
• Quantitative bounds on convergence of time-inhomogeneous Markov chains

  2004cited by this paper
• Improving Asymptotic Variance of MCMC Estimators: Non-reversible Chains are Better

  2004cited by this paper
• Sufficient burn-in for Gibbs samplers for a hierarchical random effects model

  2004cited by this paper
• Geometric Convergence Rates for Time-Sampled Markov Chains

  2003cited by this paper
• Polynomial ergodicity of Markov transition kernels

  2003cited by this paper
• Central Limit Theorem for Markov Processes

  2003cited by this paper
• Computable bounds for V-geometric ergodicity of Markov transition kernels

  2003cited by this paper
• Asymptotic Variance and Convergence Rates of Nearly-Periodic MCMC Algorithms

  2002cited by this paper
• A review of asymptotic convergence for general state space Markov chains

  2002cited by this paper
• On the applicability of regenerative simulation in Markov chain Monte Carlo

  2002cited by this paper
• Polynomial convergence rates of Markov chains.

  2002cited by this paper
• QUANTITATIVE CONVERGENCE RATES OF MARKOV CHAINS: A SIMPLE ACCOUNT

  2002cited by this paper
• Corrigendum to : Bounds on regeneration times and convergence rates for Markov chains

  2001cited by this paper
• Markov Chains and De‐initializing Processes

  2001cited by this paper
• Geometric renewal convergence rates from hazard rates

  2001cited by this paper
• Optimal scaling for various Metropolis-Hastings algorithms

  2001influential reference
• Markov Chains and De-initialising Processes by

  2001cited by this paper
• Honest Exploration of Intractable Probability Distributions via Markov Chain Monte Carlo

  2001cited by this paper
• SMALL AND PSEUDO-SMALL SETS FOR MARKOV CHAINS

  2001cited by this paper
• A First Look at Rigorous Probability Theory

  2000influential reference
• Coupling, stationarity, and regeneration

  2000cited by this paper
• Perfect simulation using dominating processes on ordered spaces, with application to locally stable point processes

  2000influential reference
• Extension of Fill's perfect rejection sampling algorithm to general chains

  2000cited by this paper
• From metropolis to diffusions: Gibbs states and optimal scaling

  2000cited by this paper
• Possible biases induced by MCMC convergence diagnostics

  1999cited by this paper
• A note on acceptance rate criteria for CLTS for Metropolis–Hastings algorithms

  1999cited by this paper
• Optimal metropolis algorithms for product measures on the vertices of a hypercube

  1998cited by this paper
• Markov‐chain monte carlo: Some practical implications of theoretical results

  1998cited by this paper
• A simulation approach to convergence rates for Markov chain Monte Carlo algorithms

  1998cited by this paper
• Log Gaussian Cox Processes

  1998cited by this paper
• Optimal scaling of discrete approximations to Langevin diffusions

  1998cited by this paper
• Weak convergence and optimal scaling of random walk Metropolis algorithms

  1997cited by this paper
• Shift-coupling and convergence rates of ergodic averages

  1997cited by this paper
• Geometric Ergodicity and Hybrid Markov Chains

  1997influential reference
• Markov Chain Monte Carlo in Practice

  1997cited by this paper
• Geometric convergence and central limit theorems for multidimensional Hastings and Metropolis algorithms

  1996influential reference
• Rates of convergence of the Hastings and Metropolis algorithms

  1996cited by this paper
• Computable exponential convergence rates for stochastically ordered Markov processes

  1996cited by this paper
• ON THE CONVERGENCE OF THE MARKOV CHAIN SIMULATION METHOD

  1996cited by this paper
• Exact sampling with coupled Markov chains and applications to statistical mechanics

  1996cited by this paper
• Convergence Rates for Markov Chains

  1995cited by this paper
• RATES OF CONVERGENCE FOR GIBBS SAMPLING FOR VARIANCE COMPONENT MODELS

  1995cited by this paper
• Minorization Conditions and Convergence Rates for Markov Chain Monte Carlo

  1995cited by this paper
• Regeneration in Markov chain samplers

  1995cited by this paper
• Markov Chains for Exploring Posterior Distributions

  1994cited by this paper
• Computable Bounds for Geometric Convergence Rates of Markov Chains

  1994cited by this paper
• Inequalities for rare events in time-reversible Markov chains II

  1993cited by this paper
• A slowly mixing Markov chain with implications for Gibbs sampling

  1993cited by this paper
• Markov Chains and Stochastic Stability

  1993influential reference
• Bayesian computation via the gibbs sampler and related markov chain monte carlo methods (with discus

  1993cited by this paper
• Rates of Convergence for Data Augmentation on Finite Sample Spaces

  1993cited by this paper
• Probability: Theory and Examples

  1993cited by this paper
• Inference from Iterative Simulation Using Multiple Sequences

  1992cited by this paper
• Practical Markov Chain Monte Carlo

  1992cited by this paper
• Improved Bounds for Mixing Rates of Markov Chains and Multicommodity Flow

  1992cited by this paper
• Lectures on the Coupling Method

  1992cited by this paper
• Sampling-Based Approaches to Calculating Marginal Densities

  1990cited by this paper
• Applied Probability and Queues

  1989influential reference
• Group representations in probability and statistics

  1988cited by this paper
• Strong uniform times and finite random walks

  1987cited by this paper
• Applied Probability and Queues

  1987cited by this paper
• Central limit theorem for additive functionals of reversible Markov processes and applications to simple exclusions

  1986cited by this paper
• General irreducible Markov chains and non-negative operators: Embedded renewal processes

  1984influential reference
• Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images

  1984cited by this paper
• A New Approach to the Limit Theory of Recurrent Markov Chains

  1978cited by this paper
• Uniform and ratio limit theorems for Markov renewal and semi-regenerative processes on a general state space

  1978cited by this paper
• On coupling of Markov chains

  1976cited by this paper
• A bound for the error in the normal approximation to the distribution of a sum of dependent random variables

  1972cited by this paper
• Lecture Notes on Limit Theorems for Markov Chain Transition Probabilities

  1971cited by this paper
• Independent and stationary sequences of random variables

  1971influential reference
• Monte Carlo Sampling Methods Using Markov Chains and Their Applications

  1970cited by this paper
• Contributions to Doeblin's theory of Markov processes

  1967cited by this paper
• A Central Limit Theorem for a Class of Dependent Random Variables

  1963cited by this paper
• The Lindeberg-Lévy theorem for martingales

  1961influential reference
• Stochastic processes

  1953cited by this paper
• Diagnostics: A Comparative Review

  year unknowncited by this paper

• Reverse Diffusion Sampling with Tensor Train Approximations of Hamilton-Jacobi-Bellman Equations

  2026cites this paper
• Higher-Order Hit-&-Run Samplers for Linearly Constrained Densities

  2026influential citation
• Ergodicity of an Adaptive MCMC Sampler under a Probability Bound

  2026cites this paper
• Solving Poisson's equation for Wasserstein contractive Markov chains

  2026cites this paper
• On randomized step sizes in Metropolis-Hastings algorithms

  2026influential citation
• Geometric ergodicity of Gibbs samplers for linear latent models with GIG variance mixtures

  2026cites this paper
• Entropic Mirror Monte Carlo

  2026cites this paper
• On micromodes in Bayesian posterior distributions and their implications for MCMC

  2026cites this paper
• Delayed Acceptance Elliptical Slice Sampling

  2025cites this paper
• Stochastic Process of Magneto-Photo-Thermoelastic Waves in Semiconductor Materials with the Change in Electrical Conductivity

  2025cites this paper
• Synchronized step multilevel Markov chain Monte Carlo

  2025cites this paper
• Sampling with time-changed Markov processes

  2025cites this paper
• SOReL and TOReL: Two Methods for Fully Offline Reinforcement Learning

  2025cites this paper
• Sampling nodes and hyperedges via random walks on large hypergraphs

  2025cites this paper
• Differential privacy guarantees of Markov chain Monte Carlo algorithms

  2025cites this paper
• Computable bounds for solutions to Poisson’s equation and perturbation of Markov kernels

  2025cites this paper
• Accelerated Parallel Tempering via Neural Transports

  2025cites this paper
• Generalized Guarantees for Variational Inference in the Presence of Even and Elliptical Symmetry

  2025cites this paper
• Parallel computations for Metropolis Markov chains with Picard maps

  2025cites this paper
• Geometric Ergodicity for Skip-Free Markov Processes and Its Applications to Queueing Models

  2025cites this paper
• The diameter of a stochastic matrix: A new measure for sensitivity analysis in Bayesian networks

  2025cites this paper
• MatBYIB: A MATLAB-Based Toolkit for Parameter Estimation of Eccentric Gravitational Waves from EMRIs

  2025cites this paper
• Worst-case mixing estimates for Brownian motion with semipermeable barriers

  2025influential citation
• A fast non-reversible sampler for Bayesian finite mixture models

  2025influential citation
• Convergence of the adapted empirical measure for mixing observations

  2025cites this paper
• Likelihood level adapted estimation of marginal likelihood for Bayesian model selection

  2025cites this paper
• Delayed Acceptance Slice Sampling

  2025cites this paper
• Autocovariance and Optimal Design for Random Walk Metropolis-Hastings Algorithm

  2025cites this paper
• Constrained Dikin-Langevin diffusion for polyhedra

  2025cites this paper
• Back-testing credit risk parameters on low default portfolios: a simple Bayesian transfer learning approach with an application to sovereign risk‖

  2025cites this paper
• Randomised Postiterations for Calibrated BayesCG

  2025cites this paper
• Computable bounds on convergence of Markov chains in Wasserstein distance via contractive drift

  2025cites this paper
• Global central limit theorems for Markov chains

  2025cites this paper
• Asymptotically exact variational flows via involutive MCMC kernels

  2025cites this paper
• AURA: Precise Abstract Interpretation of Probabilistic Programs with Interval Data Uncertainty

  2025cites this paper
• On the variance reduction of Hamiltonian Monte Carlo via an approximation scheme

  2025cites this paper
• Nonasymptotic bounds for forward processes in denoising diffusions: Ornstein–Uhlenbeck is hard to beat

  2025cites this paper
• A Stochastic Differential Equation Model for HIVAIDS Transmission Dynamics in Heterosexual Populations

  2025cites this paper
• Cascading and Proxy Membership Inference Attacks

  2025cites this paper
• Optimizing Data Center Energy Consumption Using Machine Learning and Monte Carlo Simulation

  2025influential citation
• Variational Inference with Mixtures of Isotropic Gaussians

  2025cites this paper
• Limits of predictability in top-N recommendation

  2024cites this paper
• Contraction of Markovian Operators in Orlicz Spaces and Error Bounds for Markov Chain Monte Carlo

  2024cites this paper
• A Mathematical Model for the Impact of 3HP and Social Programme Implementation on the Incidence and Mortality of Tuberculosis: Study in Brazil

  2024influential citation
• When are Unbiased Monte Carlo Estimators More Preferable than Biased Ones?

  2024influential citation
• Bayesian penalized empirical likelihood and Markov Chain Monte Carlo sampling

  2024cites this paper
• Peer-to-peer energy trading of solar and energy storage: A networked multiagent reinforcement learning approach

  2024cites this paper
• AdvNF: Reducing Mode Collapse in Conditional Normalising Flows using Adversarial Learning

  2024cites this paper
• A mathematical model for the study of HIV/AIDS transmission with PrEP coverage increase and parameter estimation using MCMC with a Bayesian approach

  2024influential citation
• On importance sampling and independent Metropolis-Hastings with an unbounded weight function

  2024influential citation
• On Doeblin Curves and Their Properties

  2024cites this paper
• Constrained Sampling with Primal-Dual Langevin Monte Carlo

  2024cites this paper
• Generative Modelling with Tensor Train approximations of Hamilton-Jacobi-Bellman equations

  2024cites this paper
• Graph-accelerated Markov Chain Monte Carlo using Approximate Samples

  2024cites this paper
• Dimension-free mixing times of Gibbs samplers for Bayesian hierarchical models

  2024cites this paper
• Approximate Counting of Linear Extensions in Practice

  2024cites this paper
• Statistical and Geometrical properties of regularized Kernel Kullback-Leibler divergence

  2024cites this paper
• MCMC-Net: accelerating Markov Chain Monte Carlo with neural networks for inverse problems

  2024cites this paper
• Comparing the Efficiency of General State Space Reversible MCMC Algorithms

  2024cites this paper
• Jump Restore Light Transport

  2024cites this paper
• Non-asymptotic bounds for forward processes in denoising diffusions: Ornstein-Uhlenbeck is hard to beat

  2024cites this paper
• Irreducibility of nonsmooth state-space models with an application to CMA-ES

  2024cites this paper
• Large Language Models as Markov Chains

  2024cites this paper
• AutoStep: Locally adaptive involutive MCMC

  2024cites this paper
• Convergence Diagnostics for Entity Resolution

  2024cites this paper
• Networked Multiagent Reinforcement Learning for Peer-to-Peer Energy Trading

  2024cites this paper
• Generating Interior Images with Latent User Preferences Through GANs

  2024cites this paper
• Less Interaction with Forward Models in Langevin Dynamics: Enrichment and Homotopy

  2024cites this paper
• The diameter of a stochastic matrix: A new measure for sensitivity analysis in Bayesian networks

  2024cites this paper
• Towards Adopting Intelligent Livestock Tracking System: An Exploratory Study

  2024cites this paper
• Simplifying Deep Temporal Difference Learning

  2024influential citation
• Is Gibbs sampling faster than Hamiltonian Monte Carlo on GLMs?

  2024cites this paper
• Uniform Sampling of Negative Edge Weights in Shortest Path Networks

  2024influential citation
• Theoretical Guarantees for Variational Inference with Fixed-Variance Mixture of Gaussians

  2024cites this paper
• Unbiased Markov Chain Monte Carlo: what, why, and how

  2024influential citation
• Theoretical guarantees for lifted samplers

  2024cites this paper
• Gaussian Process Regression‐Based Bayesian Optimisation (G‐BO) of Model Parameters—A WRF Model Case Study of Southeast Australia Heat Extremes

  2024cites this paper
• Multivariate strong invariance principle and uncertainty assessment for time in-homogeneous cyclic MCMC samplers

  2024cites this paper
• Provable Convergence and Limitations of Geometric Tempering for Langevin Dynamics

  2024cites this paper
• Ai-Sampler: Adversarial Learning of Markov kernels with involutive maps

  2024cites this paper
• Generative vs. Discriminative modeling under the lens of uncertainty quantification

  2024cites this paper
• Markovian Flow Matching: Accelerating MCMC with Continuous Normalizing Flows

  2024cites this paper
• Statistical Inference for Privatized Data with Unknown Sample Size

  2024cites this paper
• Robustness to Model Approximation, Model Learning From Data, and Sample Complexity in Wasserstein Regular MDPs

  2024influential citation
• Explicit bounds for spectral theory of geometrically ergodic Markov kernels and applications

  2024cites this paper
• Lightweight Feature-based Priority Sampling for Industrial IoT Multivariate Time Series

  2023cites this paper
• Geometric Ergodicity for Hamiltonian Monte Carlo on Compact Manifolds

  2023cites this paper
• Predicting response to combination evofosfamide and immunotherapy under hypoxic conditions in murine models of colon cancer

  2023cites this paper
• Metropolis-adjusted interacting particle sampling

  2023influential citation
• A methodology of phenotyping ICU patients from EHR data: High-fidelity, personalized, and interpretable phenotypes estimation

  2023cites this paper
• Designing Long-term Group Fair Policies in Dynamical Systems

  2023cites this paper
• Learning from small data sets: Patch‐based regularizers in inverse problems for image reconstruction

  2023cites this paper
• Neural parameter calibration and uncertainty quantification for epidemic forecasting

  2023cites this paper
• Selecting Hyperparameters of Nonlinear Support Vector Machine Using Bayesian Inference

  2023cites this paper
• STANLEY: Stochastic Gradient Anisotropic Langevin Dynamics for Learning Energy-Based Models

  2023cites this paper
• Causal structure learning with momentum: Sampling distributions over Markov Equivalence Classes of DAGs

  2023cites this paper
• General Equilibrium in a Heterogeneous-Agent Incomplete-Market Economy with Many Consumption Goods and a Risk-Free Bond

  2023cites this paper
• Irreducibility of Markov Chains on simplicial complexes, the Spectrum of the Discrete Hodge Laplacian and Homology

  2023cites this paper
• Quantitative approximation of the invariant distribution of a Markov chain. A new approach

  2023cites this paper
• A calculus for Markov chain Monte Carlo: studying approximations in algorithms

  2023cites this paper