Autonomous transfer for reinforcement learning

Recent work in transfer learning has succeeded in making reinforcement learning algorithms more efficient by incorporating knowledge from previous tasks. However, such methods typically must be provided either a full model of the tasks or an explicit relation mapping one task into the other. An autonomous agent may not have access to such high-level information, but would be able to analyze its experience to find similarities between tasks. In this paper we introduce Modeling Approximate State Transitions by Exploiting Regression (MASTER), a method for automatically learning a mapping from one task to another through an agent's experience. We empirically demonstrate that such learned relationships can significantly improve the speed of a reinforcement learning algorithm in a series of Mountain Car tasks. Additionally, we demonstrate that our method may also assist with the difficult problem of task selection for transfer.

• Publication year

  2008

• Venue

  Adaptive Agents and Multi-Agent Systems

• Publication date

  2008-05-12

• Fields of study

  Computer Science

• Identifiers
  DOI 10.65109/iryx6371
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Transfer Learning in Real-Time Strategy Games Using Hybrid CBR/RL

  2007cited by this paper
• Graph-Based Domain Mapping for Transfer Learning in General Games

  2007influential reference
• Transfer Learning via Inter-Task Mappings for Temporal Difference Learning

  2007influential reference
• Transfer via inter-task mappings in policy search reinforcement learning

  2007influential reference
• Transfer Learning in Reinforcement Learning Problems Through Partial Policy Recycling

  2007cited by this paper
• Cross-domain transfer for reinforcement learning

  2007cited by this paper
• An Experts Algorithm for Transfer Learning

  2007influential reference
• Probabilistic policy reuse in a reinforcement learning agent

  2006cited by this paper
• Kernel-Based Reinforcement Learning

  2006cited by this paper
• Value-Function-Based Transfer for Reinforcement Learning Using Structure Mapping

  2006influential reference
• Using Homomorphisms to Transfer Options across Continuous Reinforcement Learning Domains

  2006influential reference
• General Game Playing: Overview of the AAAI Competition

  2005cited by this paper
• Giving Advice about Preferred Actions to Reinforcement Learners Via Knowledge-Based Kernel Regression

  2005cited by this paper
• Using Advice to Transfer Knowledge Acquired in One Reinforcement Learning Task to Another

  2005cited by this paper
• Reinforcement learning with replacing eligibility traces

  2004cited by this paper
• Evolving Neural Networks through Augmenting Topologies

  2002cited by this paper
• Relational reinforcement learning

  2001cited by this paper
• Data mining

  1997cited by this paper
• On-line Q-learning using connectionist systems

  1994influential reference
• Case-Based Reasoning: Foundational Issues, Methodological Variations, and System Approaches

  1994cited by this paper
• Brains, behavior, and robotics

  1981influential reference
• An efficient method for finding the minimum of a function of several variables without calculating derivatives

  1964cited by this paper

• Transfer Learning for Dynamical Systems Models via Autoencoders and GANs

  2024cites this paper
• Three Dogmas of Reinforcement Learning

  2024cites this paper
• Knowledge Transfer for Cross-Domain Reinforcement Learning: A Systematic Review

  2024cites this paper
• Federated transfer learning for intrusion detection system in industrial iot 4.0

  2024cites this paper
• Dataset Distillation for Offline Reinforcement Learning

  2024cites this paper
• Learning state-action correspondence across reinforcement learning control tasks via partially paired trajectories

  2024influential citation
• Similarity-based Knowledge Transfer for Cross-Domain Reinforcement Learning

  2023cites this paper
• Multi-step motion learning by combining learning-from-demonstration and policy-search

  2023cites this paper
• Evolving Intertask Mappings for Transfer in Reinforcement Learning

  2023influential citation
• Cross-domain adaptive transfer reinforcement learning based on state-action correspondence

  2022cites this paper
• Lifelong Inverse Reinforcement Learning

  2022cites this paper
• A taxonomy for similarity metrics between Markov decision processes

  2022influential citation
• Body Calibration: Automatic Inter-Task Mapping between Multi-Legged Robots with Different Embodiments in Transfer Reinforcement Learning

  2022cites this paper
• George: Learning to Place Long-Lived Containers in Large Clusters with Operation Constraints

  2021cites this paper
• Federated Transfer Learning for IIoT Devices With Low Computing Power Based on Blockchain and Edge Computing

  2021cites this paper
• A Taxonomy of Similarity Metrics for Markov Decision Processes

  2021cites this paper
• Transferring Domain Knowledge with an Adviser in Continuous Tasks

  2021cites this paper
• Blind Spot Detection for Safe Sim-to-Real Transfer

  2020cites this paper
• FIRM: An Intelligent Fine-Grained Resource Management Framework for SLO-Oriented Microservices

  2020cites this paper
• A conceptual framework for externally-influenced agents: an assisted reinforcement learning review

  2020cites this paper
• Few-Shot Learning

  2020cites this paper
• Latent Structure Matching for Knowledge Transfer in Reinforcement Learning

  2020cites this paper
• Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey

  2020cites this paper
• Transfer Learning in Reinforcement Learning

  2020cites this paper
• Direct Policy Search Reinforcement Learning Based on Variational Bayesian Inference

  2020cites this paper
• Privacy-Preserving Transfer Learning

  2020cites this paper
• Feature-Based Transfer Learning

  2020cites this paper
• Transfer Learning in Dialogue Systems

  2020cites this paper
• REPAINT: Knowledge Transfer in Deep Reinforcement Learning

  2020cites this paper
• AutoTL: Learning to Transfer Automatically

  2020cites this paper
• Model-based reinforcement learning with missing data

  2020cites this paper
• Model-Based Transfer Learning

  2020cites this paper
• Transfer Learning for Efficient Iterative Safety Validation

  2020cites this paper
• REPAINT: Knowledge Transfer in Deep Actor-Critic Reinforcement Learning

  2020cites this paper
• Transfer Learning in Recommender Systems

  2020cites this paper
• Semi-Supervised Fine-Grained Image Categorization Using Transfer Learning With Hierarchical Multi-Scale Adversarial Networks

  2019cites this paper
• Kernel-Based Direct Policy Search Reinforcement Learning Based on Variational Bayesian Inference

  2019cites this paper
• Transfer Learning in Natural Language Processing

  2019cites this paper
• Dynamic heuristic acceleration of linearly approximated SARSA(λ\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\lambda

  2019cites this paper
• Towards Self-constructive Artificial Intelligence: Algorithmic basis (Part I)

  2019cites this paper
• Decentralized Reinforcement Learning of Robot Behaviors

  2018influential citation
• Adversarial Transfer Learning

  2018cites this paper
• Transfer reinforcement learning for task-oriented dialogue systems

  2018cites this paper
• Q-Learning Acceleration via State-Space Partitioning

  2018cites this paper
• Discovering multi-purpose modules through deep multitask learning

  2018cites this paper
• Scaling Genetic Programming to Challenging Reinforcement Tasks through Emergent Modularity

  2018cites this paper
• Direct Policy Search Reinforcement Learning Based on Variational Bayesian Inference

  2018cites this paper
• Cross-domain Dialogue Policy Transfer via Simultaneous Speech-act and Slot Alignment

  2018cites this paper
• Discovering Agent Behaviors Through Code Reuse: Examples From Half-Field Offense and Ms. Pac-Man

  2018cites this paper
• Heterogeneous transfer learning

  2017cites this paper
• Faster Reinforcement Learning Using Active Simulators

  2017cites this paper
• Towards Zero-Shot Autonomous Inter-Task Mapping through Object-Oriented Task Description

  2017cites this paper
• An autonomous inter-task mapping learning method via artificial neural network for transfer learning

  2017cites this paper
• Transfer learning via linear multi-variable mapping under reinforcement learning framework

  2017cites this paper
• Scalable transfer learning in heterogeneous, dynamic environments

  2017influential citation
• Decoupled Learning of Environment Characteristics for Safe Exploration

  2017cites this paper
• Perturbation Training for Human-Robot Teams

  2017cites this paper
• Experience Learning From Basic Patterns for Efficient Robot Navigation in Indoor Environments

  2017cites this paper
• Fast Path Planning Using Experience Learning from Obstacle Patterns

  2016cites this paper
• Source Task Creation for Curriculum Learning

  2016cites this paper
• Policy Derivation Methods for Critic-Only Reinforcement Learning in Continuous Action Spaces

  2016cites this paper
• Adaptive value function approximation in reinforcement learning using wavelets

  2016cites this paper
• Autonomous Agents and Multiagent Systems

  2016cites this paper
• Policy Improvement Methods for Reinforcement Learning in Continuous Spaces

  2016cites this paper
• Using Awareness to Promote Richer, More Human-Like Behaviors in Artificial Agents

  2016cites this paper
• Decentralized Reinforcement Learning Applied to Mobile Robots

  2016influential citation
• Representation discovery using a fixed basis in reinforcement learning

  2016cites this paper
• RLLib: C++ Library to Predict, Control, and Represent Learnable Knowledge Using On/Off Policy Reinforcement Learning

  2015cites this paper
• Autonomous Cross-Domain Knowledge Transfer in Lifelong Policy Gradient Reinforcement Learning

  2015cites this paper
• Regularized feature selection in reinforcement learning

  2015cites this paper
• Transfer learning with probabilistic mapping selection

  2015cites this paper
• Towards inherently adaptive first person shooter agents using reinforcement learning

  2015cites this paper
• Transferring knowledge as heuristics in reinforcement learning: A case-based approach

  2015cites this paper
• Unsupervised Cross-Domain Transfer in Policy Gradient Reinforcement Learning via Manifold Alignment

  2015cites this paper
• Policy Transfer using Reward Shaping

  2015cites this paper
• Transitive Transfer Learning

  2015cites this paper
• Reuse of Neural Modules for General Video Game Playing

  2015cites this paper
• On the Cross-Domain Reusability of Neural Modules for General Video Game Playing

  2015cites this paper
• Learnable knowledge for autonomous agents

  2015cites this paper
• Studying Constrained Clustering Problems Using Homotopy Maps

  2015cites this paper
• Hierarchical Functional Concepts for Knowledge Transfer among Reinforcement Learning Agents

  2015cites this paper
• An Autonomous Transfer Learning Algorithm for TD-Learners

  2014cites this paper
• Reinforcement learning transfer based on subgoal discovery and subtask similarity

  2014cites this paper
• Learning Transfer Automatic through Data Mining in Reinforcement Learning

  2014cites this paper
• Solution of Constrained Clustering Problems through Homotopy Tracking

  2014cites this paper
• Online Multi-Task Learning for Policy Gradient Methods

  2014cites this paper
• Learning parameterized motor skills on a humanoid robot

  2014cites this paper
• Convergence estimation utilizing fractal dimensional analysis for reinforcement learning

  2013cites this paper
• Transferring task models in Reinforcement Learning agents

  2013cites this paper
• Probability-one Homotopy Maps for Constrained Clustering Problems

  2013cites this paper
• Automated transfer in reinforcement learning

  2013cites this paper
• Adaptive reservoir computing through evolution and learning

  2013cites this paper
• Automatically Mapped Transfer between Reinforcement Learning Tasks via Three-Way Restricted Boltzmann Machines

  2013cites this paper
• Probability-one homotopy maps for tracking constrained clustering solutions

  2013cites this paper
• Transfer in Reinforcement Learning via Shared Features

  2012cites this paper
• Symbol Generation and Grounding for Reinforcement Learning Agents Using Affordances and Dictionary Compression

  2012cites this paper
• Transfer in Reinforcement Learning via Shared Features Citation

  2012cites this paper
• Policy transfer in mobile robots using neuro-evolutionary navigation

  2012cites this paper
• Discovering Continuous Homomorphisms for Transfer

  2012cites this paper
• Reinforcement learning transfer via sparse coding

  2012cites this paper