Limit Theory for Controlled McKean-Vlasov Dynamics

This paper rigorously connects the problem of optimal control of McKean-Vlasov dynamics with large systems of interacting controlled state processes. Precisely, the empirical distributions of near-optimal control-state pairs for the $n$-state systems, as $n$ tends to infinity, admit limit points in distribution (if the objective functions are suitably coercive), and every such limit is supported on the set of optimal control-state pairs for the McKean-Vlasov problem. Conversely, any distribution on the set of optimal control-state pairs for the McKean-Vlasov problem can be realized as a limit in this manner. Arguments are based on controlled martingale problems, which lend themselves naturally to existence proofs; along the way it is shown that a large class of McKean-Vlasov control problems admit optimal Markovian controls.

• Publication year

  2016

• Venue

  SIAM Journal of Control and Optimization

• Publication date

  2016-09-26

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1137/16M1095895arXiv 1609.08064
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Dynamic Programming for Optimal Control of Stochastic McKean-Vlasov Dynamics

  2016cited by this paper
• Continuous time mean-variance portfolio optimization through the mean field approach

  2016cited by this paper
• Randomized dynamic programming principle and Feynman-Kac representation for optimal control of McKean-Vlasov dynamics

  2016cited by this paper
• Bellman equation and viscosity solutions for mean-field stochastic control problem

  2015cited by this paper
• The Master Equation for Large Population Equilibriums

  2014cited by this paper
• Dynamic Programming for Mean-Field Type Control

  2014cited by this paper
• The Master equation in mean field theory

  2014cited by this paper
• A general characterization of the mean field limit for stochastic differential games

  2014cited by this paper
• Existence of optimal controls for systems governed by mean-field stochastic differential equations

  2014cited by this paper
• Mean field games with common noise

  2014cited by this paper
• On the connection between symmetric $N$-player games and mean field games

  2014cited by this paper
• Mean field games via controlled martingale problems: Existence of Markovian equilibria

  2014cited by this paper
• Mean Field Games and Mean Field Type Control Theory

  2013cited by this paper
• Forward-Backward Stochastic Differential Equations and Controlled McKean Vlasov Dynamics

  2013cited by this paper
• Capacities, Measurable Selection and Dynamic Programming Part II: Application in Stochastic Control Problems

  2013cited by this paper
• Weak and strong solutions of general stochastic models

  2013cited by this paper
• On the existence of strict optimal controls for constrained, controlled Markov processes in continuous time

  2012cited by this paper
• Control of McKean–Vlasov dynamics versus mean field games

  2012cited by this paper
• Propagation of chaos for rank-based interacting diffusions and long time behaviour of a scalar quasilinear parabolic equation

  2012cited by this paper
• Social Optima in Mean Field LQG Control: Centralized and Decentralized Strategies

  2012cited by this paper
• A Maximum Principle for SDEs of Mean-Field Type

  2011cited by this paper
• A General Stochastic Maximum Principle for SDEs of Mean-field Type

  2011cited by this paper
• Large deviation properties of weakly interacting processes via weak convergence methods

  2010influential reference
• Large systems of diffusions interacting through their ranks

  2010cited by this paper
• Mimicking an Itô process by a solution of a stochastic differential equation

  2010cited by this paper
• Introduction to stochastic partial differential equations

  2008cited by this paper
• Mean field games

  2007cited by this paper
• Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle

  2006cited by this paper
• Markov Processes: Characterization and Convergence

  2005cited by this paper
• TOPICS IN OPTIMAL TRANSPORTATION (Graduate Studies in Mathematics 58) By CÉDRIC VILLANI: 370 pp., US$59.00, ISBN 0-8218-3312-X (American Mathematical Society, Providence, RI, 2003)

  2004cited by this paper
• Topics in Optimal Transportation

  2003cited by this paper
• Individual and mass behaviour in large population stochastic wireless power control problems: centralized and Nash equilibrium solutions

  2003cited by this paper
• Existence of Markov Controls and Characterization of Optimal Markov Controls

  1998influential reference
• Propagation of chaos and fluctuations for a moderate model with smooth initial data

  1998cited by this paper
• Stochastic particle approximations for generalized Boltzmann models and convergence estimates

  1997cited by this paper
• Asymptotic behaviour of some interacting particle systems; McKean-Vlasov and Boltzmann models

  1996cited by this paper
• Ecole d'été de probabilités de Saint-Flour XIX, 1989

  1991cited by this paper
• Weak Limit Theorems for Stochastic Integrals and Stochastic Differential Equations

  1991cited by this paper
• Martingale measures and stochastic calculus

  1990influential reference
• On the McKean‐Vlasov Limit for Interacting Diffusions

  1988cited by this paper
• A propagation of chaos result for a system of particles with moderate interaction

  1987cited by this paper
• Compactification methods in the control of degenerate diffusions: existence of an optimal control

  1987cited by this paper
• An introduction to stochastic partial differential equations

  1986cited by this paper
• Mimicking the one-dimensional marginal distributions of processes having an ito differential

  1986cited by this paper
• A certain class of diffusion processes associated with nonlinear parabolic equations

  1984cited by this paper
• A Martingale Approach to the Law of Large Numbers for Weakly Interacting Stochastic Processes

  1984cited by this paper
• Multidimensional Diffusion Processes

  1979cited by this paper
• On the existence of optimal controls

  1970cited by this paper
• Existence of optimal controls

  1967influential reference
• Zeitschrift fur Wahrscheinlichkeitstheorie und Verwandte Gebiete.

  1963cited by this paper
• On Certain Questions in the Theory of Optimal Control

  1962influential reference

• Scalable Method for Mean Field Control with Kernel Interactions via Random Fourier Features

  2026influential citation
• Stochastic Control Methods for Optimization

  2026cites this paper
• Marginal flows of non-entropic weak Schr\"odinger bridges

  2025cites this paper
• Mean-field optimal control with stochastic leaders

  2025cites this paper
• Sparse stabilization of mean-field agent dynamics through a three-operator splitting method

  2025cites this paper
• Robust mean-field control under common noise uncertainty

  2025cites this paper
• A non-exchangeable mean field control problem with controlled interactions

  2025cites this paper
• Approximately optimal distributed controls for high-dimensional stochastic systems with pairwise interaction through controls

  2025cites this paper
• Near Optimality of Discrete-Time Approximations for Controlled McKean-Vlasov Diffusions and Interacting Particle Systems

  2025influential citation
• Conditional McKean-Vlasov control

  2025cites this paper
• Error estimates for finite-dimensional approximations of Hamilton-Jacobi-Bellman equations on the Wasserstein space

  2025cites this paper
• Limit theory for mean-field control problems with common noise adapted controls

  2025cites this paper
• Balancing Efficiency and Complexity in Large Scale Multi-Agent Optimization: A Reconfigurable Mean Field Game Approach

  2025cites this paper
• Mean field control with absorption

  2025cites this paper
• Mean field social optimization: feedback person-by-person optimality and the dynamic programming equation

  2025cites this paper
• McKean-Vlasov equations with rough common noise

  2025cites this paper
• A notion of BSDE on the Wasserstein space and its applications to control problems and PDEs

  2025cites this paper
• Extended mean field game theoretical optimal distributed control for large scale multi-agent systems: An efficiency-complexity tradeoff

  2025cites this paper
• Mean Field Control with Poissonian Common Noise: A Pathwise Compactification Approach

  2025influential citation
• Probabilistic Analysis of Graphon Mean Field Control

  2025cites this paper
• Particle Approximation for Conditional Control with Soft Killing

  2025cites this paper
• Infinite Time Horizon Optimal Control of McKean-Vlasov SDEs

  2025cites this paper
• Extended mean field control: a finite-dimensional numerical approximation

  2025cites this paper
• Mean Field Game of Controls with State Reflections: Existence and Limit Theory

  2025influential citation
• Constrained mean-field control with singular control: Existence, stochastic maximum principle and constrained FBSDE

  2025cites this paper
• Polynomial McKean-Vlasov SDEs

  2025cites this paper
• Weak Solutions to the Master Equation of Potential Mean Field Games

  2025cites this paper
• On McKean-Vlasov Branching Diffusion Processes

  2024cites this paper
• A particle system approach towards the global well-posedness of master equations for potential mean field games of control

  2024cites this paper
• Regularity and stability for the Gibbs conditioning principle on path space via McKean-Vlasov control

  2024cites this paper
• Score-based Neural Ordinary Differential Equations for Computing Mean Field Control Problems

  2024cites this paper
• Convergence rate for random walk approximations of mean field BSDEs

  2024cites this paper
• Control of conditional processes and Fleming–Viot dynamics

  2024influential citation
• Optimal control problems with generalized mean-field dynamics and viscosity solution to Master Bellman equation

  2024cites this paper
• Finite dimensional projections of HJB equations in the Wasserstein space

  2024cites this paper
• Convergence Rate of Particle System for Second-Order PDEs on Wasserstein Space

  2024cites this paper
• Mean-field control of non exchangeable systems

  2024cites this paper
• Mean-Field Control for Diffusion Aggregation system with Coulomb Interaction

  2024cites this paper
• Robust decentralized control of coupled systems via risk sensitive control of decoupled or simple models with measure change

  2024influential citation
• Decentralized Exchangeable Stochastic Dynamic Teams in Continuous-time, their Mean-Field Limits and Optimality of Symmetric Policies

  2024cites this paper
• Asymptotic Nash Equilibria of Finite-State Ergodic Markovian Mean Field Games

  2024cites this paper
• A comparison principle for semilinear Hamilton–Jacobi–Bellman equations in the Wasserstein space

  2024cites this paper
• Convergence analysis of controlled particle systems arising in deep learning: from finite to infinite sample size

  2024cites this paper
• Optimality of Symmetric Independent Policies under Decentralized Mean-Field Information Sharing for Stochastic Teams and Equivalence with McKean-Vlasov Control of a Representative Agent

  2024cites this paper
• Convergence of the Deep Galerkin Method for Mean Field Control Problems

  2024cites this paper
• Large-Population Optimal Control with Mixed Agents: The Multi-Scale Analysis and Decentralized Control

  2023cites this paper
• Sharp convergence rates for mean field control in the region of strong regularity

  2023influential citation
• On Time‐Inconsistency in Mean‐Field Games

  2023cites this paper
• Set-valued propagation of chaos for controlled path-dependent McKean-Vlasov SPDEs

  2023influential citation
• Well-posedness of Hamilton-Jacobi equations in the Wasserstein space: non-convex Hamiltonians and common noise

  2023influential citation
• Time-inconsistent mean-field stopping problems: a regularised equilibrium approach

  2023cites this paper
• A Viscosity Solution Theory of Stochastic Hamilton-Jacobi-Bellman equations in the Wasserstein Space

  2023cites this paper
• Controlled Interacting Branching Diffusion Processes: Relaxed Formulation in the Mean-Field Regime

  2023cites this paper
• A limit theory for controlled McKean-Vlasov SPDEs

  2023influential citation
• On the limit theory of mean field optimal stopping with non-Markov dynamics and common noise

  2023cites this paper
• Weak Convergence of Stochastic Integrals on Skorokhod Space in Skorokhod's J1 and M1 Topologies

  2023cites this paper
• Infinite Horizon Average Cost Optimality Criteria for Mean-Field Control

  2023cites this paper
• Mean Field Approximation of an Optimal Control Problem for the Continuity Equation Arising in Smart Charging

  2023cites this paper
• A comparison principle for semilinear Hamilton-Jacobi-Bellman equations in the Wasserstein space

  2023cites this paper
• Safe Model-Based Multi-Agent Mean-Field Reinforcement Learning

  2023cites this paper
• Approximately optimal distributed stochastic controls beyond the mean field setting

  2023cites this paper
• Continuous Time q-Learning for Mean-Field Control Problems

  2023cites this paper
• Controlled superprocesses and HJB equation in the space of finite measures

  2023influential citation
• Mean-field limit for stochastic control problems under state constraint

  2023influential citation
• Mean Field Control and Finite Agent Approximation for Regime-Switching Jump Diffusions

  2023cites this paper
• Regularity of the value function and quantitative propagation of chaos for mean field control problems

  2022influential citation
• Mean field game master equations with anti-monotonicity conditions

  2022cites this paper
• Rate of homogenization for fully-coupled McKean-Vlasov SDEs

  2022cites this paper
• An Algebraic Convergence Rate for the Optimal Control of McKean-Vlasov Dynamics

  2022influential citation
• Linear-Quadratic Mean Field Games of Controls with Non-Monotone Data

  2022cites this paper
• Pontryagin maximum principle for the deterministic mean field type optimal control problem via the Lagrangian approach

  2022cites this paper
• Quantitative propagation of chaos for mean field Markov decision process with common noise

  2022cites this paper
• Importance Sampling for the Empirical Measure of Weakly Interacting Diffusions

  2022cites this paper
• From finite population optimal stopping to mean field optimal stopping

  2022influential citation
• Indexability is Not Enough for Whittle: Improved, Near-Optimal Algorithms for Restless Bandits

  2022cites this paper
• A finite-dimensional approximation for partial differential equations on Wasserstein space

  2022cites this paper
• Finite Approximations and Q learning for Mean Field Type Multi Agent Control

  2022cites this paper
• Finite Approximations for Mean-Field Type Multi-agent Control and Their Near Optimality

  2022cites this paper
• Learning Mean Field Games: A Survey

  2022cites this paper
• Laplace principle for large population games with control interaction

  2021cites this paper
• A McKean–Vlasov Game of Commodity Production, Consumption and Trading

  2021cites this paper
• Optimal bailout strategies resulting from the drift controlled supercooled Stefan problem

  2021cites this paper
• Optimal control of a first order Fokker-Planck equation with reaction term and density constraints

  2021cites this paper
• Constrained optimal control of a first order Fokker-Planck with reaction term

  2021cites this paper
• 93E20 Optimal stochastic control 49K45 Optimality conditions for problems involving randomness 49L20 Dynamic programming in optimal control and differential games

  2021cites this paper
• Mean field control and finite dimensional approximation for regime-switching jump diffusions

  2021cites this paper
• On the Approximation of Cooperative Heterogeneous Multi-Agent Reinforcement Learning (MARL) using Mean Field Control (MFC)

  2021cites this paper
• Large population games with interactions through controls and common noise: convergence results and equivalence between open–loop and closed–loop controls

  2021influential citation
• Master Bellman equation in the Wasserstein space: Uniqueness of viscosity solutions

  2021influential citation
• Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learning

  2021cites this paper
• Closed-loop convergence for mean field games with common noise

  2021cites this paper
• Optimal Policies for Convex Symmetric Stochastic Dynamic Teams and their Mean-Field Limit

  2021cites this paper
• Mean field game of mutual holding

  2021cites this paper
• DeepSets and Their Derivative Networks for Solving Symmetric PDEs

  2021cites this paper
• Rate of convergence for particle approximation of PDEs in Wasserstein space

  2021cites this paper
• Mean field social optimization: feedback person-by-person optimality and the master equation

  2020cites this paper
• Finite State Mean Field Games with Wright-Fisher Common Noise as Limits of N-Player Weighted Games

  2020cites this paper
• Independently Randomized Symmetric Policies are Optimal for Exchangeable Stochastic Teams with Infinitely Many Decision Makers

  2020cites this paper
• Optimality of Independently Randomized Symmetric Policies for Exchangeable Stochastic Teams with Infinitely Many Decision Makers

  2020cites this paper
• Lattice approximations of the first-order mean field type differential games

  2020cites this paper