Optimal and Autonomous Control Using Reinforcement Learning: A Survey

This paper reviews the current state of the art on reinforcement learning (RL)-based feedback control solutions to optimal regulation and tracking of single and multiagent systems. Existing RL solutions to both optimal <inline-formula> <tex-math notation="LaTeX">$\mathcal {H}_{2}$ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">$\mathcal {H}_\infty $ </tex-math></inline-formula> control problems, as well as graphical games, will be reviewed. RL methods learn the solution to optimal control and game problems online and using measured data along the system trajectories. We discuss Q-learning and the integral RL algorithm as core algorithms for discrete-time (DT) and continuous-time (CT) systems, respectively. Moreover, we discuss a new direction of off-policy RL for both CT and DT systems. Finally, we review several applications.

• Publication year

  2018

• Venue

  IEEE Transactions on Neural Networks and Learning Systems

• Publication date

  2018-06-01

• Fields of study

  Medicine, Computer Science, Engineering

• Identifiers
  DOI 10.1109/TNNLS.2017.2773458PMID 29771662
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Optimal Switching of DC–DC Power Converters Using Approximate Dynamic Programming

  2018cited by this paper
• Event-Triggered Adaptive Dynamic Programming for Continuous-Time Systems With Control Constraints

  2017cited by this paper
• Adaptive Event-Triggered Control Based on Heuristic Dynamic Programming for Nonlinear Discrete-Time Systems

  2017cited by this paper
• Near Optimal Event-Triggered Control of Nonlinear Discrete-Time Systems Using Neurodynamic Programming

  2016cited by this paper
• Machine learning: Trends, perspectives, and prospects

  2015cited by this paper
• $ {H}_{ {\infty }}$ Tracking Control of Completely Unknown Continuous-Time Systems via Off-Policy Reinforcement Learning

  2015cited by this paper
• Approximate $N$ -Player Nonzero-Sum Game Solution for an Uncertain Continuous Nonlinear System

  2015cited by this paper
• Optimal Tracking Control of Unknown Discrete-Time Linear Systems Using Input-Output Measured Data

  2015cited by this paper
• Actor–Critic-Based Optimal Tracking for Partially Unknown Nonlinear Discrete-Time Systems

  2015cited by this paper
• Robust Adaptive Dynamic Programming and Feedback Stabilization of Nonlinear Systems

  2014cited by this paper
• Feedback Solution to Optimal Switching Problems With Switching Cost

  2014cited by this paper
• Off-Policy Reinforcement Learning for $ H_\infty $ Control Design

  2013cited by this paper
• Reinforcement Learning for Partially Observable Dynamic Processes: Adaptive Dynamic Programming Using Measured Output Data

  2011cited by this paper
• Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof

  2008cited by this paper
• Reinforcement Learning in Continuous Time and Space

  2000cited by this paper
• DYNAMIC PROGRAMMING AND LAGRANGE MULTIPLIERS.

  1956cited by this paper

• Reinforcement learning-driven nonsingular fast terminal sliding mode control for pipe crack sealing manipulator

  2026cites this paper
• An optimal control method for uncertain process industry based on working condition relevance and policy transfer

  2026cites this paper
• When an Approximate Model Suffices for Optimal Control

  2026cites this paper
• Asynchronous multithreading reinforcement learning with attention-based significance measurement for collision-free robot navigation

  2026cites this paper
• Beyond Monotonic Droop: Expanding Feasible Control Regions for Optimal Frequency Regulation

  2026cites this paper
• Pursuit-evasion with a focus – attention-augmented reinforcement learning for swarm-to-swarm orbital game

  2026cites this paper
• A Learning-Based Closed-Loop Fluid Flow Regulation

  2026cites this paper
• Reinforcement learning in pulse energy converters, process knowledge learned to transfer perspective framework

  2026cites this paper
• Event-triggered intelligent critic control for uncertain robotic systems: Guaranteed optimal tracking with disturbance attenuation

  2026cites this paper
• Deep reinforcement learning-based control of a 1-DOF helicopter: a comparative analysis of classical and modern methods

  2026cites this paper
• Multiplayer Cascaded Policy Iteration for Nash Differential Games

  2026cites this paper
• Input-output data-driven sensor selection for cyber-physical systems

  2026cites this paper
• A Multi-Agent Reinforcement Learning Approach to Autonomous Aircraft Taxiing with Taxiing Time, Fuel Consumption, and Emission Optimization

  2026cites this paper
• WIMLE: Uncertainty-Aware World Models with IMLE for Sample-Efficient Continuous Control

  2026cites this paper
• Non-zero-sum games in continuous-time nonlinear systems: Multiplayer cascaded solutions

  2026cites this paper
• Data-driven optimal control of nonlinear systems via Carleman operator and value iteration

  2026cites this paper
• Explainable AI with DRL for Smart Home Energy Management and Residential Cost Savings in IoT-Based Autonomic Systems

  2026cites this paper
• Control theory in the era of intelligent systems: Applications, trends, and future directions

  2026cites this paper
• Hierarchical distributed optimal cluster formation control for MASs with predefined-time convergence via hybrid zero-sum-game strategy

  2026cites this paper
• Nonlinear extended state observer-based model-free near-optimal sliding mode water level controller of an inverted U-tube steam generator

  2026cites this paper
• Prescribed-Time Observer-Based HI-RL Secure Output Tracking Control for Heterogeneous MASs Under DoS Attacks

  2026cites this paper
• A learning-based value iteration scheme for singularly perturbed systems and its application in RC ladder circuit

  2026cites this paper
• AVERY: Adaptive VLM Split Computing through Embodied Self-Awareness for Efficient Disaster Response Systems

  2025cites this paper
• Adaptive Dynamic Programming Infinite‐Horizon Optimal Tracking Control for Stochastic Linear Discrete‐Time Systems

  2025cites this paper
• A Survey on Reinforcement Learning for Optimal Decision-Making and Control of Intelligent Vehicles

  2025cites this paper
• Combined Learning and Control: A New Paradigm for Optimal Control with Unknown Dynamics

  2025cites this paper
• Decentralized Data-Driven State Feedback Control for Graph-Based Interconnected Systems

  2025cites this paper
• Stabilization of Nonlinear Systems with State-Dependent Representation: From Model-Based to Direct Data-Driven Control

  2025cites this paper
• Model‐Free Approximate Dynamic Programming for Stochastic Zero‐Sum Games: Algorithm Design and Analysis

  2025cites this paper
• An adaptive extremum-seeking control approach to reinforcement learning

  2025cites this paper
• Sliding Mode-Based Regulation Control for Nonlinear System with Disturbances Via Reinforcement Learning

  2025cites this paper
• Snake robots: A state-of-the-art review on design, locomotion, control, and real-world applications

  2025cites this paper
• Model-based reinforcement learning for active flow control

  2025cites this paper
• Reinforcement Learning-Based Robust Optimal Attitude Tracking Control for Quadrotor Unmanned Aerial Vehicle with Prescribed Performance

  2025cites this paper
• Input-Output Data-Driven Sensor Selection for Cyber-Physical Systems

  2025cites this paper
• Reinforcement Learning for Linear Optimal Control Using a Modified Newton-Kleinman Method

  2025cites this paper
• A Review of Machine‐Learning‐Based Control Methods for Permanent Magnet Synchronous Machines

  2025cites this paper
• Formal Verification of Neural Certificates Done Dynamically

  2025cites this paper
• Link Augmentation and Q-Learning for Set Stabilization in Switched Boolean Networks

  2025cites this paper
• Reinforcement learning in pursuit-evasion differential game: safety, stability and robustness

  2025cites this paper
• Leveraging Analytic Gradients in Provably Safe Reinforcement Learning

  2025cites this paper
• The state of hybrid artificial intelligence for interstellar missions

  2025cites this paper
• Hardware-Software Stitching Algorithm in Lightweight Q-Learning System on Chip (SoC) for Shortest Path Optimization

  2025cites this paper
• Learning finite-horizon optimal control with unknown control-affine dynamics

  2025cites this paper
• Hybrid actuators and their reuse methodologies for amphibious robots

  2025cites this paper
• Reinforcement learning‐based degradation‐tolerant control design for affine nonlinear systems

  2025cites this paper
• Model-free Optimal Formation Control Strategy for Multiple Quadrotors

  2025cites this paper
• Approximate optimal trajectory tracking and dynamic obstacle avoidance for affine system via online learning

  2025cites this paper
• Reinforcement Learning-Based Nonlinear Optimal Discrete-Time Control of Power Systems

  2025cites this paper
• LLM-Driven Pareto-Optimal Multi-Mode Reinforcement Learning for Adaptive UAV Navigation in Urban Wind Environments

  2025cites this paper
• Embodied AI: From LLMs to World Models [Feature]

  2025cites this paper
• Lie-algebra Learning for Mobile Robots Tracking Control with Model Uncertainty

  2025cites this paper
• Model-Based and Model-Free Algorithms for Stochastic $H_{\infty}$ Tracking Control

  2025cites this paper
• On Suboptimal Safety-Critical Tracking Controller Design

  2025cites this paper
• Research Status and Development Trends of Deep Reinforcement Learning in the Intelligent Transformation of Agricultural Machinery

  2025cites this paper
• Homotopic reinforcement learning for distributed consensus control of stochastic Markov jump multi-agent systems

  2025cites this paper
• Towards Certified Safe Personalization in Learning-Enabled Human-in-the-loop Human-in-the-plant Systems

  2025cites this paper
• Practical Intelligent Automatic Train Operations which Optimize Switching Times Autonomously

  2025cites this paper
• A Fully Data-Driven Value Iteration for Stochastic LQR: Convergence, Robustness and Stability

  2025cites this paper
• Learning-Based Geometric Tracking Control for Rigid Body Dynamics

  2025cites this paper
• Output regulation for aperiodic sampled linear delay systems via impulsive feedback

  2025cites this paper
• Intelligent attitude control in tunnel boring machine considering uncertain underground environment using deep reinforcement learning

  2025cites this paper
• Bridging Continuous-time LQR and Reinforcement Learning via Gradient Flow of the Bellman Error

  2025cites this paper
• Resource-constrained dynamic planning and model-free control in turbulent urban environments

  2025cites this paper
• Mechanism-data-driven control strategy for active suspension systems: Integrating deep reinforcement learning with differential geometry to enhance vehicle ride comfort

  2025cites this paper
• Reinforcement Learning-Based Fault-Tolerant Control of Uncertain Strict-Feedback Nonlinear Systems With Intermittent Actuator Faults

  2025cites this paper
• Enhancing generalization in reinforcement learning for robotic control through deep policy similarities

  2025cites this paper
• Enhancing offline reinforcement learning for wastewater treatment via transition filter and prioritized approximation loss

  2025cites this paper
• Adaptive neural optimal control for pure‐feedback nonlinear systems with prescribed performance and error compensating mechanism

  2025cites this paper
• A plug-and-play fully on-the-job real-time reinforcement learning algorithm for a direct-drive tandem-wing experiment platforms under multiple random operating conditions

  2025cites this paper
• Data-driven control for stochastic linear-quadratic optimal problem with completely unknown dynamics

  2025cites this paper
• Symbiotic Control of Uncertain Dynamical Systems: Harnessing Synergy Between Fixed-Gain Control and Adaptive Learning Architectures

  2025cites this paper
• Adjustable behavior-guided adaptive dynamic programming for neural learning control

  2025cites this paper
• Adaptive Actor-Critic Based Optimal Regulation for Drift-Free Nonlinear Systems

  2025cites this paper
• Enhancing PID control robustness in CSTRs: a hybrid approach to tuning under external disturbances with GA, PSO, and machine learning

  2025cites this paper
• A tutorial review of policy iteration methods in reinforcement learning for nonlinear optimal control

  2025cites this paper
• Improved DDPG based on enhancing decision evaluation for path planning in high-density environments

  2025cites this paper
• Strategic Learning for Disturbance Rejection in Multi‐Agent Systems: Nash and Minmax in Graphical Games

  2025cites this paper
• Robust Model-Free Reinforcement Learning Based Current Control of PMSM Drives

  2025cites this paper
• Trajectory tracking of a SCARA robot using intelligent active force control

  2025cites this paper
• Geometric Fault-Tolerant Neural Network Tracking Control of Unknown Systems on Matrix Lie Groups

  2025cites this paper
• On-Policy and Off-Policy Value Iteration Algorithms for Stochastic Zero-Sum Dynamic Games

  2025cites this paper
• Two-component controller design to safeguard data-driven predictive control

  2025cites this paper
• Safe Physics-Informed Machine Learning for Optimal Predefined-Time Stabilization: A Lyapunov-Based Approach

  2025cites this paper
• A Physics‐Informed Learning Framework to Solve the Infinite‐Horizon Optimal Control Problem

  2025cites this paper
• Impact of cyber physical systems on enhancing robotic system autonomy: a brief critical review

  2025cites this paper
• Task Adaptation from Skills: Information Geometry, Disentanglement, and New Objectives for Unsupervised Reinforcement Learning

  2025cites this paper
• Data-Driven Control for High-Speed Interceptors With Nonuniformly Sampled Data

  2025cites this paper
• On the Robustness of Derivative-free Methods for Linear Quadratic Regulator

  2025cites this paper
• Concurrent shape and controller optimization of autonomous underwater vehicles for path following tasks using evolutionary algorithm-driven reinforcement learning

  2025cites this paper
• Model-free optimal control for discrete-time Markovian jump linear systems: A Q-learning approach

  2025cites this paper
• Optimizing Wastewater Treatment Control Using Policy-Constrained Offline Reinforcement Learning and Lyapunov Density Model

  2025cites this paper
• Nonlinear and RL-Based Controllers for Inverted Pendulum Systems

  2025cites this paper
• Model-free stochastic linear quadratic design by semidefinite programming

  2025cites this paper
• Adaptive Weighting by Sinkhorn Distance for Sharing Experiences between Multi-Task Reinforcement Learning in Sparse-reward Environments

  2025cites this paper
• Case Studies of Generative Machine Learning Models for Dynamical Systems

  2025cites this paper
• Improving long-term stability of optoelectronic oscillators using genetic algorithm-optimized dual-parameter control.

  2025cites this paper
• Control Oriented Reinforcement Learning: A Survey of Recent Progress and Applications

  2025cites this paper
• Model-Free Derivative Feedback Control for Active Magnetic Levitation System

  2025cites this paper
• Transition Control of a Double-Inverted Pendulum System Using Sim2Real Reinforcement Learning

  2025cites this paper