Evolutionary Algorithms for Reinforcement Learning

There are two distinct approaches to solving reinforcement learning problems, namely, searching in value function space and searching in policy space. Temporal difference methods and evolutionary algorithms are well-known examples of these approaches. Kaelbling, Littman and Moore recently provided an informative survey of temporal difference methods. This article focuses on the application of evolutionary algorithms to the reinforcement learning problem, emphasizing alternative policy representations, credit assignment methods, and problem-specific genetic operators. Strengths and weaknesses of the evolutionary approach to reinforcement learning are presented, along with a survey of representative applications.

• Publication year

  1999

• Venue

  Journal of Artificial Intelligence Research

• Publication date

  1999-07-01

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1613/jair.613arXiv 1106.0221
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Reinforcement Learning: An Introduction

  1998cited by this paper
• Introduction to Reinforcement Learning

  1998cited by this paper
• Probably approximately correct (PAC) learning analysis

  1997cited by this paper
• Forming Neural Networks Through Efficient and Adaptive Coevolution

  1997cited by this paper
• Robot Shaping: An Experiment in Behavior Engineering

  1997influential reference
• The design and analysis of a computational model of cooperative coevolution

  1997influential reference
• Proportional selection and sampling algorithms

  1997cited by this paper
• Reinforcement Learning: A Survey

  1996cited by this paper
• Evolving Obstacle Avoidance Behavior in a Robot Arm

  1996influential reference
• Reinforcement learning with selective perception and hidden state

  1996cited by this paper
• Continual learning in reinforcement environments

  1995cited by this paper
• Learning Sequential Decision

  1995influential reference
• A Coevolutionary Approach to Learning Sequential Decision Rules

  1995cited by this paper
• Circle in the round: State space attractors for evolved sighted robots

  1995cited by this paper
• Escaping brittleness: the possibilities of general-purpose learning algorithms applied to parallel rule-based systems

  1995influential reference
• Learning from delayed rewards

  1995cited by this paper
• ROBOT LEARNING WITH PARALLEL GENETIC ALGORITHMS ON NETWORKED COMPUTERS

  1995influential reference
• Evolving Neural Networks to Focus Minimax Search

  1994cited by this paper
• Sequential Behavior and Learning in Evolved Dynamical Neural Networks

  1994cited by this paper
• Simple Genetic Algorithms with Linear Fitness

  1994cited by this paper
• LEARNING ROBOT BEHAVIORS USING GENETIC ALGORITHMS

  1994cited by this paper
• ZCS: A Zeroth Level Classifier System

  1994influential reference
• Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence, by John H. Holland MIT Press (Bradford Books), Cambridge, Mass., 1992, xiv+211 pp. (Paperback £13.50, cloth £26.95)

  1993influential reference
• Genetic Algorithms for Tracking Changing Environments

  1993cited by this paper
• Explorations in Evolutionary Robotics

  1993cited by this paper
• Using a Genetic Algorithm to Learn Behaviors for Autonomous Vehicles

  1992cited by this paper
• Memory Approaches to Reinforcement Learning in Non-Markovian Domains

  1992cited by this paper
• Genetic Programming: On the Programming of Computers by Means of Natural Selection

  1992cited by this paper
• An Approach to Anytime Learning

  1992cited by this paper
• Genetic Algorithms for Changing Environments

  1992cited by this paper
• Reinforcement Learning with Perceptual Aliasing: The Perceptual Distinctions Approach

  1992cited by this paper
• Selection in Massively Parallel Genetic Algorithms

  1991cited by this paper
• The Development of a World Class Othello Program

  1990cited by this paper
• Evolving networks: using the genetic algorithm with connectionist learning

  1990cited by this paper
• Learning and Sequential Decision Making

  1989cited by this paper
• The GENITOR Algorithm and Selection Pressure: Why Rank-Based Allocation of Reproductive Trials is Best

  1989influential reference
• A Study of Control Parameters Affecting Online Performance of Genetic Algorithms for Function Optimization

  1989cited by this paper
• Learning to control an inverted pendulum using neural networks

  1989cited by this paper
• Genitor: a different genetic algorithm

  1988influential reference
• Genetic Algorithms in Search Optimization and Machine Learning

  1988cited by this paper
• Genetic Algorithms and Classifier Systems: Foundations and Future Directions

  1987influential reference
• Genetic Algorithms with Sharing for Multimodalfunction Optimization

  1987cited by this paper
• Optimization of Control Parameters for Genetic Algorithms

  1986cited by this paper
• Multi-Objective Learning via Genetic Algorithms

  1985cited by this paper
• A theory of the learnable

  1984cited by this paper
• Flexible Learning of Problem Solving Heuristics Through Adaptive Search

  1983influential reference
• COGNITIVE SYSTEMS BASED ON ADAPTIVE ALGORITHMS1

  1978influential reference
• Cognitive systems based on adaptive algorithms

  1977influential reference
• Adaptation in Natural and Artificial Sys-tems: An Introductory Analysis with Applications to Biology

  1975cited by this paper
• An Analysis Of The Behavior Of A Class Of Genetic Adaptive Systems

  1975cited by this paper
• Artificial Intelligence through Simulated Evolution

  1966cited by this paper

• CMA-MAPPO: Integrating Covariance Matrix Adaptation Evolution Strategy with Multi-Agent Proximal Policy Optimization for enhanced exploration in sparse-reward environments

  2026cites this paper
• Dual-Mind World Models: A General Framework for Learning in Dynamic Wireless Networks

  2025cites this paper
• Toward Safety-First Human-Like Decision Making for Autonomous Vehicles in Time-Varying Traffic Flow

  2025cites this paper
• Discovering Control Scheduler Policies Through Reinforcement Learning and Evolutionary Strategies

  2025cites this paper
• Evolutionary Policy Optimization

  2025cites this paper
• Randomised Optimism via Competitive Co-Evolution for Matrix Games with Bandit Feedback

  2025cites this paper
• GenAI-based Multi-Agent Reinforcement Learning towards Distributed Agent Intelligence: A Generative-RL Agent Perspective

  2025cites this paper
• Beyond game environments: Evolutionary algorithms with parameter space noise for task-oriented dialogue policy exploration

  2025cites this paper
• The state of hybrid artificial intelligence for interstellar missions

  2025cites this paper
• Quality-Diversity Driven Action Swarm Evolution in Reinforcement Learning

  2025cites this paper
• ERL-ER-FedHG: Evolutionary reinforcement learning with experience replay for federated heterogeneous graphs

  2025cites this paper
• Exploiting Multi‐Objective Reinforcement Learning and Explainable Artificial Intelligence to Navigate Robust Regional Water Supply Investment Pathways

  2025cites this paper
• Hardware-aware Decision Tree Ensembles for Radar Data Processing on the Edge with Online Learning

  2025cites this paper
• AI-enabled routing in next generation networks: A survey

  2025cites this paper
• Методика построения управления космическими аппаратами с использованием методов обучения с подкреплением

  2025cites this paper
• Improving energy efficiency and scalable computing in Federated Learning

  2025cites this paper
• Reinforcement Learning-Based Autonomous Control of a Spacecraft in the Solar Gravitational Lens’ Focus

  2025cites this paper
• Reinforcement learning in densely recurrent biological networks

  2025cites this paper
• Benchmarking MOEAs for solving continuous multi-objective RL problems

  2025cites this paper
• Minimisation of Quasar-Convex Functions Using Random Zeroth-Order Oracles

  2025cites this paper
• CosmoCore-Evo: Evolutionary Dream-Replay Reinforcement Learning for Adaptive Code Generation

  2025cites this paper
• Learning Action Primitives in Manipulation Robots

  2025cites this paper
• Eventually, all you need is a simple evolutionary algorithm (for neuroevolution of continuous control policies)

  2024cites this paper
• From Bayes to Darwin: evolutionary search as an exaptation from sampling-based Bayesian inference

  2024cites this paper
• An Alternating Proximal Gradient Algorithm for Nonsmooth Nonconvex-Linear Minimax Problems with Coupled Linear Constraints

  2024cites this paper
• Emergent Discovery of Reinforced Programs using Q-Learning and Planning: A Proof of Concept

  2024cites this paper
• Continuously evolving rewards in an open-ended environment

  2024cites this paper
• Modeling Air Combat Behavior for Simulation-Based Pilot Training: A Survey of Machine Learning Approaches

  2024cites this paper
• Evolutionary Reinforcement Learning via Cooperative Coevolution

  2024cites this paper
• Joint Transmit and Jamming Power Optimization for Secrecy in Energy Harvesting Networks: A Reinforcement Learning Approach

  2024cites this paper
• Evolutionary Multi-agent Reinforcement Learning in Group Social Dilemmas

  2024cites this paper
• A new evolutionary strategy for reinforcement learning

  2024cites this paper
• Spacecraft Control Design Methodology Based on Reinforcement Learning Methods

  2024cites this paper
• Efficient Elitist Cooperative Evolutionary Algorithm for Multi-Objective Reinforcement Learning

  2023cites this paper
• Evolving Intertask Mappings for Transfer in Reinforcement Learning

  2023cites this paper
• Novelty Search for Neuroevolutionary Reinforcement Learning of Deceptive Systems: An Application to Control of Colloidal Self-assembly

  2023cites this paper
• Evolutionary Sampling Agent for Expensive Problems

  2023cites this paper
• Applications of Reinforcement Learning for maintenance of engineering systems: A review

  2023cites this paper
• Autonomy and Intelligence in the Computing Continuum: Challenges, Enablers, and Future Directions for Orchestration

  2022cites this paper
• Automatic Parameter Optimization Using Genetic Algorithm in Deep Reinforcement Learning for Robotic Manipulation Tasks

  2022cites this paper
• Surrogate-assisted distributed swarm optimisation for computationally expensive geoscientific models

  2022cites this paper
• Iterative Minimax Games with Coupled Linear Constraints

  2022cites this paper
• Surrogate-assisted distributed swarm optimisation for computationally expensive models

  2022cites this paper
• AI-powered model repair: an experience report—lessons learned, challenges, and opportunities

  2022cites this paper
• Optimizing the dynamic treatment regime of in-hospital warfarin anticoagulation in patients after surgical valve replacement using reinforcement learning

  2022cites this paper
• More Precise Runtime Analyses of Non-elitist Evolutionary Algorithms in Uncertain Environments

  2022cites this paper
• Machine Learning Based Load Balancing Algorithms in Future Heterogeneous Networks: A Survey

  2022cites this paper
• Automatic Parameter Optimization Using Genetic Algorithm in Deep Reinforcement Learning for Robotic Manipulation Tasks

  2022cites this paper
• Improving Behavioural Cloning with Human-Driven Dynamic Dataset Augmentation

  2022cites this paper
• Evolutionary Action Selection for Gradient-based Policy Learning

  2022cites this paper
• A review on reinforcement learning algorithms and applications in supply chain management

  2022cites this paper
• A heuristically accelerated reinforcement learning method for maintenance policy of an assembly line

  2022cites this paper
• Rule-based reinforcement learning methodology to inform evolutionary algorithms for constrained optimization of engineering applications

  2021cites this paper
• Does Lifelong Learning Affect Mobile Robot Evolution?

  2021cites this paper
• Derivative-free reinforcement learning: a review

  2021cites this paper
• Evolving hierarchical memory-prediction machines in multi-task reinforcement learning

  2021cites this paper
• How RL Agents Behave When Their Actions Are Modified

  2021cites this paper
• A Review on Deep Reinforcement Learning for the management of SDN and NFV in Edge-IoT

  2021cites this paper
• Finding Simple Solutions to Multi-Task Visual Reinforcement Learning Problems with Tangled Program Graphs

  2021cites this paper
• Deep multiagent reinforcement learning: challenges and directions

  2021cites this paper
• Internet of Things for Environment Protection and Sustainable Living

  2021cites this paper
• NEORL: NeuroEvolution Optimization with Reinforcement Learning

  2021cites this paper
• Reinforcement Learning based Optimization of Bayesian Networks for Generating Feasible Vehicle Configuration Suggestions

  2021cites this paper
• Multiagent Deep Reinforcement Learning: Challenges and Directions Towards Human-Like Approaches

  2021cites this paper
• Performance of Implicit Stochastic Approaches to the Synthesis of Multireservoir Operating Rules

  2020cites this paper
• Localization Uncertainty-driven Adaptive Framework for Controlling Ground Vehicle Robots

  2020cites this paper
• Automatic guided vehicles (AGVs) in the rail transit intelligent manufacturing environment

  2020cites this paper
• A Novel Update Mechanism for Q-Networks Based On Extreme Learning Machines

  2020cites this paper
• Momentum accelerates evolutionary dynamics

  2020cites this paper
• Fitness Landscape Features and Reward Shaping in Reinforcement Learning Policy Spaces

  2020cites this paper
• Introduction

  2020cites this paper
• Zeroth-Order Algorithms for Nonconvex Minimax Problems with Improved Complexities

  2020cites this paper
• Comparison of competing market mechanisms with reinforcement learning in a carpooling scenario

  2020cites this paper
• Zeroth-order algorithms for nonconvex–strongly-concave minimax problems with improved complexities

  2020cites this paper
• A Framework for Reinforcement Learning and Planning

  2020cites this paper
• A Unifying Framework for Reinforcement Learning and Planning

  2020cites this paper
• Efficient admission control and resource allocation mechanisms for public safety communications over 5G network slice

  2019cites this paper
• A Reinforcement Learning Based Intercell Interference Coordination in LTE Networks

  2019cites this paper
• Evolutionary Algorithm for Adaptive Quantum-Channel Control

  2019influential citation
• Deep Reinforcement Learning Using Genetic Algorithm for Parameter Optimization

  2019cites this paper
• Reusability and Transferability of Macro Actions for Reinforcement Learning

  2019cites this paper
• Improving the Performance of Evolutionary Algorithms via Gradient-Based Initialization

  2019cites this paper
• Construction of Macro Actions for Deep Reinforcement Learning

  2019cites this paper
• Human-Centered Reinforcement Learning: A Survey

  2019cites this paper
• Concepts of Artificial Intelligence for Computer-Assisted Drug Discovery.

  2019cites this paper
• On the Value of ENSO State for Urban Water Supply System Operators: Opportunities, Trade‐Offs, and Challenges

  2019cites this paper
• The Evolution of Reinforcement Learning *

  2019cites this paper
• Reinforcement Learning: A Technical Introduction – Part I

  2019cites this paper
• AGCS Technique to Improve the Performance of Neural Networks

  2019cites this paper
• Deep Learning with Evolutionary Strategies for Building Autonomous Agents Behaviour

  2019cites this paper
• Interpretable Reinforcement Learning Policies by Evolutionary Computation

  2019cites this paper
• Reinforcement learning versus evolutionary computation: A survey on hybrid algorithms

  2019cites this paper
• Training Agents using Upside-Down Reinforcement Learning

  2019cites this paper
• Hybrid Mixture Model Based on a Hybrid Optimization for Spectrum Sensing to Improve the Performance of MIMO–OFDM Systems

  2019cites this paper
• Reinforcement Learning Driven Heuristic Optimization

  2019cites this paper
• ES-MAML: Simple Hessian-Free Meta Learning

  2019cites this paper
• Performance of evolutionary wavelet neural networks in acrobot control tasks

  2019cites this paper
• Self-adaptive and self-organised planning and decision-making for multi-robot systems

  2019cites this paper
• Epoch-incremental Dyna-learning and prioritized sweeping algorithms

  2018cites this paper
• Slice as an Evolutionary Service: Genetic Optimization for Inter-Slice Resource Management in 5G Networks

  2018cites this paper