Bellman equation and viscosity solutions for mean-field stochastic control problem

We consider the stochastic optimal control problem of McKean-Vlasov stochastic differential equation where the coefficients may depend upon the joint law of the state and control. By using feedback controls, we reformulate the problem into a deterministic control problem with only the marginal distribution of the process as controlled state variable, and prove that dynamic programming principle holds in its general form. Then, by relying on the notion of differentiability with respect to pro\-bability measures recently introduced by P.L. Lions in [32], and a special It{\^o} formula for flows of probability measures, we derive the (dynamic programming) Bellman equation for mean-field stochastic control problem, and prove a veri\-fication theorem in our McKean-Vlasov framework. We give explicit solutions to the Bellman equation for the linear quadratic mean-field control problem, with applications to the mean-variance portfolio selection and a systemic risk model. We also consider a notion of lifted visc-sity solutions for the Bellman equation, and show the viscosity property and uniqueness of the value function to the McKean-Vlasov control problem. Finally, we consider the case of McKean-Vlasov control problem with open-loop controls and discuss the associated dynamic programming equation that we compare with the case of closed-loop controls.

• Publication year

  2015

• Venue

  arXiv: Probability

• Publication date

  2015-12-24

• Fields of study

  Mathematics

• Identifiers
  DOI 10.1051/COCV/2017019arXiv 1512.07866
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• On time-inconsistent stochastic control in continuous time

  2017cited by this paper
• Stochastic optimal control in infinite dimension : dynamic programming and HJB equations

  2017cited by this paper
• Continuous time mean-variance portfolio optimization through the mean field approach

  2016influential reference
• Time Inconsistent Stochastic Control in Continuous Time: Theory and Examples

  2016cited by this paper
• Randomized dynamic programming principle and Feynman-Kac representation for optimal control of McKean-Vlasov dynamics

  2016influential reference
• Discrete Time McKean–Vlasov Control Problem: A Dynamic Programming Approach

  2015cited by this paper
• Metric viscosity solutions of Hamilton–Jacobi equations depending on local slopes

  2015cited by this paper
• On The Interpretation Of The Master Equation

  2015cited by this paper
• Mean-field stochastic differential equations and associated PDEs

  2014influential reference
• Linear-Quadratic Mean Field Games

  2014cited by this paper
• Dynamic Programming for Mean-Field Type Control

  2014influential reference
• The Master equation in mean field theory

  2014influential reference
• The Master Equation for Large Population Equilibriums

  2014influential reference
• Linear-Quadratic Optimal Control Problems for Mean-Field Stochastic Differential Equations

  2013cited by this paper
• Forward-Backward Stochastic Differential Equations and Controlled McKean Vlasov Dynamics

  2013cited by this paper
• Mean Field Games and Systemic Risk

  2013influential reference
• Control of McKean–Vlasov dynamics versus mean field games

  2012cited by this paper
• A Maximum Principle for SDEs of Mean-Field Type

  2011influential reference
• A Linear-Quadratic Optimal Control Problem for Mean-Field Stochastic Differential Equations

  2011influential reference
• A General Stochastic Maximum Principle for SDEs of Mean-field Type

  2011cited by this paper
• A General Theory of Markovian Time Inconsistent Stochastic Control Problems

  2010cited by this paper
• The viscosity solutions approach to stochastic control problems

  2009cited by this paper
• A Comparison Principle for Hamilton–Jacobi Equations Related to Controlled Gradient Flows in Infinite Dimensions

  2009cited by this paper
• Optimal Transport: Old and New

  2008cited by this paper
• Hamilton-Jacobi equations in the Wasserstein space

  2008cited by this paper
• Nonlinear SDEs driven by L\'evy processes and related PDEs

  2007influential reference
• Mean field games

  2007cited by this paper
• Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle

  2006cited by this paper
• Gradient Flows: In Metric Spaces and in the Space of Probability Measures

  2005cited by this paper
• Hamilton-Jacobi Equations

  2003cited by this paper
• Continuous-Time Mean-Variance Portfolio Selection: A Stochastic LQ Framework

  2000influential reference
• Controlled markov processes and viscosity solutions

  1994influential reference
• Controlled Markov processes and viscosity solutions

  1992cited by this paper
• Topics in propagation of chaos

  1991influential reference
• Continuous martingales and Brownian motion

  1990cited by this paper
• Viscosity solutions of fully nonlinear second-order equations and optimal stochastic control in infinite dimensions. III. Uniqueness of viscosity solutions for general second-order equations

  1989cited by this paper
• Viscosity solutions of fully nonlinear second-order equations and optimal stochastic control in infinite dimensions. Part I: the case of bounded stochastic evolutions

  1988cited by this paper
• Foundations of Kinetic Theory

  1956cited by this paper

• Comparison of viscosity solutions for a class of non-linear PDEs on the space of finite nonnegative measures

  2026cites this paper
• Mean-field forward-backward stochastic variational inequalities with oblique subgradients and nonlocal parabolic variational inequalities

  2026cites this paper
• Sparse optimal control in the Wasserstein space

  2026cites this paper
• A Hilbert space method to LQ optimal control of conditional McKean-Vlasov dynamics

  2026cites this paper
• Approximately optimal distributed controls for high-dimensional stochastic systems with pairwise interaction through controls

  2025cites this paper
• Convergence Rates of Time Discretization in Extended Mean Field Control

  2025cites this paper
• Mean-Field Control for Diffusion Aggregation Equation with Coulomb Interaction

  2025cites this paper
• Contracting a crowd of heterogeneous agents

  2025cites this paper
• A notion of BSDE on the Wasserstein space and its applications to control problems and PDEs

  2025cites this paper
• Global Well-posedness of Hamilton-Jacobi Equations for Linear-Quadratic Mean Field Control Problems

  2025cites this paper
• Probabilistic Analysis of Graphon Mean Field Control

  2025cites this paper
• Weak Solutions to the Master Equation of Potential Mean Field Games

  2025cites this paper
• Optimal Control of McKean--Vlasov Branching Diffusion Processes

  2025influential citation
• Ergodic Control of McKean-Vlasov Systems on the Wasserstein Space

  2025cites this paper
• Adaptive Q-learning for Path Planning of AUV in Obstacle-Dense Environments

  2025cites this paper
• Stochastic McKean-Vlasov Control Problem with Regime-Switching and Its Applications

  2025cites this paper
• Infinite Time Horizon Optimal Control of McKean-Vlasov SDEs

  2025cites this paper
• A large multi-agent system with noise both in position and control

  2025cites this paper
• Linear-quadratic optimal control for non-exchangeable mean-field SDEs and applications to systemic risk

  2025cites this paper
• Time-optimal problem in the space of probabilities measures

  2025cites this paper
• Stochastic Differential Games of Mean-Field Dynamics and Second-Order Bellman–Isaacs Equations on the Wasserstein Space

  2025influential citation
• Ergodicity and turnpike properties of linear-quadratic mean field control problems

  2025cites this paper
• A randomisation method for mean-field control problems with common noise

  2025cites this paper
• Optimal stabilisation control for discrete-time mean-field stochastic Markov jump system

  2025cites this paper
• Sufficient stochastic maximum principle for mean-field control problems with regime switching in an infinite horizon

  2025cites this paper
• On time-inconsistent extended mean-field control problems with common noise

  2024cites this paper
• A randomisation method for mean-field control problems with common noise

  2024cites this paper
• Optimal control problems with generalized mean-field dynamics and viscosity solution to Master Bellman equation

  2024cites this paper
• Finite dimensional projections of HJB equations in the Wasserstein space

  2024cites this paper
• Extended mean field control problems with constraints: The generalized Fritz-John conditions and Lagrangian method

  2024cites this paper
• Mean-Field Control for Diffusion Aggregation system with Coulomb Interaction

  2024cites this paper
• Classical solution of path-dependent mean-field semilinear PDEs

  2024cites this paper
• Well-posedness for Hamilton–Jacobi equations on the Wasserstein space on graphs

  2024cites this paper
• Viscosity solutions for mean field optimal switching with a two-time-scale Markov chain

  2024cites this paper
• A Global Optimality Principle for Fully Coupled Mean-field Control Systems

  2024cites this paper
• Infinite horizon McKean-Vlasov FBSDEs and applications to mean field control problems

  2024cites this paper
• Optimal Stopping of Branching Diffusion Processes

  2024cites this paper
• Convergence of the Deep Galerkin Method for Mean Field Control Problems

  2024cites this paper
• Hamilton–Jacobi equations for controlled gradient flows: cylindrical test functions

  2023cites this paper
• Sparse Mean Field Load Balancing in Large Localized Queueing Systems

  2023cites this paper
• Global Well-Posedness of First-Order Mean Field Games and Master Equations with Nonlinear Dynamics

  2023influential citation
• A Viscosity Solution Theory of Stochastic Hamilton-Jacobi-Bellman equations in the Wasserstein Space

  2023cites this paper
• Finite-Horizon Optimal Control of Continuous-Time Stochastic Systems with Terminal Cost of Wasserstein Distance

  2023cites this paper
• Multi-Agent Reinforcement Learning via Mean Field Control: Common Noise, Major Agents and Approximation Properties

  2023cites this paper
• Dynamic Programming for Indefinite Stochastic McKean-Vlasov LQ Control Problem under Input Constraints

  2023cites this paper
• Mean-Field Risk-Sensitive Control with HARA Utility

  2023cites this paper
• Optimal Control Problem for Reflected McKean-Vlasov SDEs

  2023cites this paper
• Density of subalgebras of Lipschitz functions in metric Sobolev spaces and applications to Wasserstein Sobolev spaces

  2023cites this paper
• Viscosity Solutions of the Eikonal Equation on the Wasserstein Space

  2023cites this paper
• Approximation of deterministic mean field type control systems

  2023cites this paper
• Learning Decentralized Partially Observable Mean Field Control for Artificial Collective Behavior

  2023influential citation
• Stochastic recursive optimal control of McKean-Vlasov type: A viscosity solution approach

  2023influential citation
• Controlled superprocesses and HJB equation in the space of finite measures

  2023cites this paper
• Ergodic control of McKean–Vlasov SDEs and associated Bellman equation

  2023cites this paper
• Fully-coupled mean-field FBSDE and maximum principle for related optimal control problem

  2023cites this paper
• Finite Dimensional Approximations of Hamilton-Jacobi-Bellman Equations for Stochastic Particle Systems with Common Noise

  2023cites this paper
• Major-Minor Mean Field Multi-Agent Reinforcement Learning

  2023cites this paper
• Discrete-Time Mean-Field Stochastic Control with Partial Observations

  2023cites this paper
• Exact Controllability for Mean-Field Type Linear Game-Based Control Systems

  2023cites this paper
• Extended mean-field control problem with partial observation

  2022cites this paper
• Pontryagin maximum principle for the deterministic mean field type optimal control problem via the Lagrangian approach

  2022cites this paper
• Viscosity Solutions for McKean-Vlasov Control on a Torus

  2022cites this paper
• Viscosity Solutions for McKean-Vlasov Control I: one-dimensional torus

  2022cites this paper
• Forward-Backward Sweep Method for the System of HJB-FP Equations in Memory-Limited Partially Observable Stochastic Control

  2022cites this paper
• Importance Sampling for the Empirical Measure of Weakly Interacting Diffusions

  2022cites this paper
• Mean Field Games with Mean-Field-Dependent Volatility, and Associated Fully Coupled Nonlocal Quasilinear Forward-Backward Parabolic Equations

  2022cites this paper
• Viscosity Solutions for Obstacle Problems on Wasserstein Space

  2022cites this paper
• Mean-field type FBSDEs in a domination-monotonicity framework and LQ multi-level Stackelberg games

  2022cites this paper
• Pricing Bermudan Option with Variable Transaction Costs under the Information-Based Model

  2022cites this paper
• Memory-Limited Partially Observable Stochastic Control and Its Mean-Field Control Approach

  2022cites this paper
• A Survey on Large-Population Systems and Scalable Multi-Agent Reinforcement Learning

  2022cites this paper
• Density of subalgebras of Lipschitz functions in metric Sobolev spaces and applications to Wasserstein Sobolev spaces

  2022cites this paper
• Hypergraphon Mean Field Games

  2022cites this paper
• Kolmogorov equations on the space of probability measures associated to the nonlinear filtering equation: the viscosity approach

  2022cites this paper
• Near Optimality of Stochastic Control for Singularly Perturbed McKean-Vlasov Systems

  2022cites this paper
• Hamilton–Jacobi equations for controlled gradient flows: the comparison principle

  2021cites this paper
• Finite Dimensional Approximations of Hamilton-Jacobi-Bellman Equations in Spaces of Probability Measures

  2021cites this paper
• Mean-field linear-quadratic stochastic differential games

  2021cites this paper
• Rate of convergence for particle approximation of PDEs in Wasserstein space

  2021cites this paper
• Dynamic Programming Equation for the Mean Field Optimal Stopping Problem

  2021cites this paper
• Discrete-Time Mean Field Control with Environment States

  2021cites this paper
• Controllability Gramian and Kalman rank condition for mean-field control systems

  2021cites this paper
• Controlled Markov chains with non-exponential discounting and distribution-dependent costs

  2021cites this paper
• Mean Field Games with Common Noises and Conditional Distribution Dependent FBSDEs

  2021cites this paper
• Mean Field Games with Mean-Field-Dependent Volatility: A PDE Approach∗

  2021cites this paper
• Master Bellman equation in the Wasserstein space: Uniqueness of viscosity solutions

  2021cites this paper
• A Fast Iterative PDE-Based Algorithm for Feedback Controls of Nonsmooth Mean-Field Control Problems

  2021cites this paper
• Controlled measure-valued martingales: A viscosity solution approach

  2021cites this paper
• Optimal bailout strategies resulting from the drift controlled supercooled Stefan problem

  2021cites this paper
• Achieving Safe Deep Reinforcement Learning via Environment Comprehension Mechanism

  2021cites this paper
• Learning Graphon Mean Field Games and Approximate Nash Equilibria

  2021cites this paper
• A level-set approach to the control of state-constrained McKean-Vlasov equations: application to renewable energy storage and portfolio selection

  2021cites this paper
• Optimal Control Problems Governed by MFSDEs with Multiple Defaults ∗

  2021influential citation
• Bellman type strategy for the continuous time mean-variance model

  2020cites this paper
• Mean Field Optimal Control and Related Problems

  2020influential citation
• Itô’s formula for flows of measures on semimartingales

  2020cites this paper
• A closed-loop saddle point for zero-sum linear-quadratic stochastic differential games with mean-field type

  2020cites this paper
• Mean-Field Limit for a Class of Stochastic Ergodic Control Problems

  2020cites this paper
• Closed-Loop Equilibrium for Time-Inconsistent McKean-Vlasov Controlled Problem

  2020cites this paper
• McKean-Vlasov Optimal Control: Limit Theory and Equivalence Between Different Formulations

  2020cites this paper