Risk-Sensitive Optimal Control for Markov Decision Processes with Monotone Cost

The existence of an optimal feedback law is established for the risk-sensitive optimal control problem with denumerable state space. The main assumptions imposed are irreducibility and anear monotonicity condition on the one-step cost function. A solution can be found constructively using either value iteration or policy iteration under suitable conditions on initial feedback law.

• Publication year

  2002

• Venue

  Mathematics of Operations Research

• Publication date

  2002-02-01

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1287/moor.27.1.192.334
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Spectral theory and limit theorems for geometrically ergodic Markov processes

  2002cited by this paper
• Multiplicative ergodicity and large deviations for an irreducible Markov chain

  2000cited by this paper
• Risk-Sensitive Control of Discrete-Time Markov Processes with Infinite Horizon

  1999cited by this paper
• Controlled Markov chains with risk-sensitive criteria: Average cost, optimality equations, and optimal solutions

  1999cited by this paper
• Algorithms for optimization and stabilization of controlled Markov chains

  1999cited by this paper
• Value iteration and optimization of multiclass queueing networks

  1998cited by this paper
• Risk sensitive control of discrete time partially observed Markov processes with infinite horizon

  1998cited by this paper
• Risk sensitive control of finite state machines on an infinite horizon. I

  1997cited by this paper
• The policy iteration algorithm for average reward Markov decision processes with general state space

  1997cited by this paper
• The Policy Improvement Algorithm for Markov Decision Processes

  1997cited by this paper
• A Liapounov bound for solutions of the Poisson equation

  1996cited by this paper
• Risk sensitive control of Markov processes in countable state space

  1996cited by this paper
• Risk-sensitive control and dynamic games for partially observed discrete-time nonlinear systems

  1994cited by this paper
• Markov Chains and Stochastic Stability

  1993influential reference
• Asymptotic analysis of nonlinear stochastic risk-sensitive control and differential games

  1992cited by this paper
• Risk sensitive optimal control and differential games

  1992cited by this paper
• Risk-Sensitive Optimal Control

  1990cited by this paper
• A Lyapunov Bound for Solutions of Poisson's Equation

  1989cited by this paper
• Multiplicative Markov Decision Chains

  1984cited by this paper
• General irreducible Markov chains and non-negative operators: List of symbols and notation

  1984cited by this paper
• Non‐Negative Matrices and Markov Chains

  1983cited by this paper
• Probability theory: Independence, interchangeability, martingales

  1978cited by this paper
• Optimal stochastic linear systems with exponential performance criteria and their relation to deterministic differential games

  1973cited by this paper
• Risk-Sensitive Markov Decision Processes

  1972cited by this paper
• Dynamic Programming

  1957cited by this paper

• Risk-Sensitive Exponential Actor Critic

  2026cites this paper
• Stochastic Approximation and Reinforcement Learning: The Interface and a Little Beyond

  2025cites this paper
• An optimal sequence for sub-Markov decision processes with risk sensitivity

  2025cites this paper
• Bayesian Risk-Sensitive Policy Optimization For MDPs With General Loss Functions

  2025cites this paper
• Team variance optimization of n-player stochastic games with separately controlled chains

  2025cites this paper
• Quantum Algorithms for Bandits with Knapsacks with Improved Regret and Time Complexities

  2025cites this paper
• Entropic Risk Optimization in Discounted MDPs: Sample Complexity Bounds with a Generative Model

  2025cites this paper
• Risk-Averse Reinforcement Learning with Itakura-Saito Loss

  2025cites this paper
• An Actor–Critic Algorithm With Function Approximation for Risk Sensitive Cost Markov Decision Processes

  2025cites this paper
• Risk-Sensitive Variational Actor-Critic: A Model-Based Approach

  2025cites this paper
• Denumerable Markov stopping games with risk-sensitive total reward criterion

  2024cites this paper
• Discrete-time hybrid control with risk-sensitive discounted costs

  2024influential citation
• AoI-aware transmission control in real-time mmwave energy harvesting systems: a risk-sensitive reinforcement learning approach

  2024cites this paper
• Nash equilibria in risk-sensitive Markov stopping games under communication conditions

  2024cites this paper
• Risk-Sensitive Average Markov Decision Processes in General Spaces

  2024cites this paper
• Pessimism Meets Risk: Risk-Sensitive Offline Reinforcement Learning

  2024influential citation
• Data-Driven Direct Adaptive Risk-Sensitive Control of Stochastic Systems

  2024cites this paper
• A Discount Vanishing Approximation for Markov Decision Processes with Risk Sensitivity

  2024cites this paper
• Risk-sensitive discounted Markov decision processes with unbounded reward functions and Borel spaces

  2024cites this paper
• Contractive approximations in average Markov decision chains driven by a risk-seeking controller

  2023cites this paper
• Markov Decision Processes Under Risk Sensitivity: A Discount Vanishing Approach

  2023cites this paper
• Markov decision processes with risk-sensitive criteria: an overview

  2023cites this paper
• Average criteria in denumerable semi-Markov decision chains under risk-aversion

  2023cites this paper
• Characterization of the optimal average cost in Markov decision chains driven by a risk-seeking controller

  2023cites this paper
• Exponential TD Learning: A Risk-Sensitive Actor-Critic Reinforcement Learning Algorithm

  2023cites this paper
• Entropic Risk Optimization in Discounted MDPs

  2023influential citation
• On the Convergence of Modified Policy Iteration in Risk-Sensitive Exponential Cost Markov Decision Processes

  2023influential citation
• Risk-Sensitive Average Optimality for Discrete-Time Markov Decision Processes

  2023cites this paper
• STEERING: Stein Information Directed Exploration for Model-Based Reinforcement Learning

  2023cites this paper
• On the global convergence of relative value iteration for infinite-horizon risk-sensitive control of diffusions

  2023cites this paper
• Ergodic risk-sensitive control - A survey

  2022cites this paper
• Risk-Sensitive Reinforcement Learning With Exponential Criteria

  2022cites this paper
• Partially observable discrete-time discounted Markov games with general utility

  2022cites this paper
• Bridging Distributional and Risk-sensitive Reinforcement Learning with Provable Regret Bounds

  2022cites this paper
• RASR: Risk-Averse Soft-Robust MDPs with EVaR and Entropic Risk

  2022influential citation
• Risk-averse policy optimization via risk-neutral policy optimization

  2022cites this paper
• Cascaded Gaps: Towards Logarithmic Regret for Risk-Sensitive Reinforcement Learning

  2022cites this paper
• Multi-Agent Reinforcement Learning with General Utilities via Decentralized Shadow Reward Actor-Critic

  2022cites this paper
• Mean-Semivariance Policy Optimization via Risk-Averse Reinforcement Learning

  2022cites this paper
• Posterior Coreset Construction with Kernelized Stein Discrepancy for Model-Based Reinforcement Learning

  2022cites this paper
• Non-stationary Risk-sensitive Reinforcement Learning: Near-optimal Dynamic Regret, Adaptive Detection, and Separation Design

  2022cites this paper
• Efficient Risk-Averse Reinforcement Learning

  2022cites this paper
• Risk-sensitive Markov stopping games with an absorbing state

  2022cites this paper
• Cascaded Gaps: Towards Gap-Dependent Regret for Risk-Sensitive Reinforcement Learning

  2022cites this paper
• Maximal dispersion of adaptive random walks

  2022cites this paper
• A Policy Gradient Algorithm for the Risk-Sensitive Exponential Cost MDP

  2022influential citation
• Discrete-time zero-sum games for Markov chains with risk-sensitive average cost criterion

  2022cites this paper
• State Augmented Constrained Reinforcement Learning: Overcoming the Limitations of Learning With Rewards

  2021cites this paper
• Ecosystem based multi-species management using Empirical Dynamic Programming

  2021cites this paper
• Nonzero-sum Risk-Sensitive Average Stochastic Games: The Case of Unbounded Costs

  2021cites this paper
• Provably Efficient Actor-Critic for Risk-Sensitive and Robust Adversarial RL: A Linear-Quadratic Case

  2021cites this paper
• Ergodic risk-sensitive control of Markov processes on countable state space revisited

  2021influential citation
• On tight bounds for function approximation error in risk-sensitive reinforcement learning

  2021cites this paper
• Risk-sensitive semi-Markov decision problems with discounted cost and general utilities

  2021cites this paper
• Risk-sensitive zero-sum stochastic differential game for jump-diffusions

  2021cites this paper
• Exponential Bellman Equation and Improved Regret Bounds for Risk-Sensitive Reinforcement Learning

  2021cites this paper
• Zero-sum semi-Markov games with a probability criterion

  2021cites this paper
• Risk-Sensitive Reinforcement Learning with Function Approximation: A Debiasing Approach

  2021cites this paper
• MARL with General Utilities via Decentralized Shadow Reward Actor-Critic

  2021cites this paper
• Stochastic Approximation with Markov Noise: Analysis and applications in reinforcement learning

  2020cites this paper
• Variational Policy Gradient Method for Reinforcement Learning with General Utilities

  2020cites this paper
• Convergence of Value Functions for Finite Horizon Markov Decision Processes with Constraints

  2020cites this paper
• Convergence of Value Functions for Finite Horizon Markov Decision Processes with Constraints

  2020cites this paper
• A Discounted Approach in Communicating Average Markov Decision Chains Under Risk-Aversion

  2020cites this paper
• Risk-Sensitive Reinforcement Learning: Near-Optimal Risk-Sample Tradeoff in Regret

  2020cites this paper
• On the relative value iteration with a risk-sensitive criterion

  2019cites this paper
• Multi-objective safe reinforcement learning

  2019cites this paper
• Multi-objective safe reinforcement learning: the relationship between multi-objective reinforcement learning and safe reinforcement learning

  2019cites this paper
• The Vanishing Discount Approach in a class of Zero-Sum Finite Games with Risk-Sensitive Average Criterion

  2019cites this paper
• Risk-sensitive control of branching processes

  2019cites this paper
• Risk-sensitive control for a class of diffusions with jumps

  2019cites this paper
• Policy Optimization for H2 Linear Control with H∞ Robustness Guarantee: Implicit Regularization and Global Convergence

  2019cites this paper
• On the policy improvement algorithm for ergodic risk-sensitive control

  2019cites this paper
• Risk-Sensitive Discounted Continuous-Time Markov Decision Processes with Unbounded Rates

  2019cites this paper
• Safe Reinforcement Learning in Continuous State Spaces

  2019cites this paper
• Discounted approximations in risk-sensitive average Markov cost chains with finite state space

  2019cites this paper
• Exit time risk-sensitive control for systems of cooperative agents

  2019cites this paper
• The study of basic risk processes by discrete-time non-homogeneous Markov processes

  2018cites this paper
• Risk-Sensitive Reinforcement Learning: A Constrained Optimization Viewpoint

  2018cites this paper
• Characterization of the Optimal Risk-Sensitive Average Cost in Denumerable Markov Decision Chains

  2018cites this paper
• Safe option-critic: learning safety in the option-critic architecture

  2018cites this paper
• Risk-Sensitive Partially Observable Markov Decision Processes as Fully Observable Multivariate Utility Optimization problems

  2018cites this paper
• Multivariate Utility Optimization with an Application to Risk-Sensitive Partially Observable Markov Decision Processes

  2018cites this paper
• Exit time risk-sensitive control for systems of cooperative agents

  2018cites this paper
• Risk Sensitive Multiple Goal Stochastic Optimization, with application to Risk Sensitive Partially Observed Markov Decision Processes

  2018cites this paper
• Quantitative Modeling of Emission Markets

  2017cites this paper
• Inverse Risk-Sensitive Reinforcement Learning

  2017cites this paper
• Particle Value Functions

  2017cites this paper
• Linear and dynamic programming approaches to degenerate risk-sensitive reward processes

  2017cites this paper
• Vanishing discount approximations in controlled Markov chains with risk-sensitive average criterion

  2017cites this paper
• Discounted approximations to the risk-sensitive average cost in finite Markov chains

  2017cites this paper
• A note on the function approximation error bound for risk-sensitive reinforcement learning

  2016cites this paper
• Nonzero-Sum Risk-Sensitive Stochastic Games on a Countable State Space

  2016influential citation
• Infinite horizon risk-sensitive control of diffusions without any blanket stability assumptions

  2016cites this paper
• Zero-sum Risk-Sensitive Stochastic Games

  2016cites this paper
• Optimal timing of living-donor liver transplantation under risk-aversion

  2016cites this paper
• On the function approximation error for risk-sensitive reinforcement learning

  2016cites this paper
• Analyzing operational risk-reward trade-offs for start-ups

  2015cites this paper
• Finite horizon risk sensitive MDP and linear programming

  2015cites this paper
• Risk-Sensitive Ergodic Control of Continuous Time Markov Processes With Denumerable State Space

  2015influential citation