Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning

No abstract is available for this paper.

• Publication year

  1999

• Venue

  Artificial Intelligence

• Publication date

  1999-08-01

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1016/S0004-3702(99)00052-1
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Artificial Intelligence in Medicine: 23rd International Conference, AIME 2025, Pavia, Italy, June 23–26, 2025, Proceedings, Part I

  2025cited by this paper
• Improved Switching among Temporally Abstract Actions". In Advances in Neural Information Processing Systems

  1999cited by this paper
• A Q-learning-based dynamic channel assignment technique for mobile communication systems

  1999cited by this paper
• Theoretical Results on Reinforcement Learning with Temporally Abstract Options

  1998cited by this paper
• Hierarchical Solution of Markov Decision Processes using Macro-actions

  1998cited by this paper
• The MAXQ Method for Hierarchical Reinforcement Learning

  1998cited by this paper
• Composing Functions to Speed up Reinforcement Learning in a Changing World

  1998cited by this paper
• Intra-Option Learning about Temporally Abstract Actions

  1998cited by this paper
• High-Level Planning and Control with Incomplete Information Using POMDP's

  1998cited by this paper
• Reinforcement Learning: An Introduction

  1998cited by this paper
• Hierarchical control and learning for markov decision processes

  1998cited by this paper
• Solving Very Large Weakly Coupled Markov Decision Processes

  1998cited by this paper
• Macro-Actions in Reinforcement Learning: An Empirical Analysis

  1998cited by this paper
• Improved Switching among Temporally Abstract Actions

  1998cited by this paper
• Reinforcement Learning with Hierarchies of Machines

  1997cited by this paper
• Modeling Agents as Qualitative Decision Makers

  1997cited by this paper
• Behaviour-based control: examples from navigation, learning, and group behaviour

  1997cited by this paper
• Multi-time Models for Temporally Abstract Planning

  1997cited by this paper
• Reinforcement Learning for Call Admission Control and Routing in Integrated Service Networks

  1997cited by this paper
• Prioritized Goal Decomposition of Markov Decision Processes: Toward a Synthesis of Classical and Decision Theoretic Planning

  1997cited by this paper
• HQ-Learning

  1997cited by this paper
• A feedback control structure for on-line learning tasks

  1997cited by this paper
• Behavior-based Control: Examples from Navigation, Learning, and Group Behavior

  1997cited by this paper
• Reinforcement Learning for Dynamic Channel Allocation in Cellular Telephone Systems

  1996cited by this paper
• Qualitative System Identification: Deriving Structure from Behavior

  1996cited by this paper
• Behavior analysis and training-a methodology for behavior engineering

  1996cited by this paper
• A Statistical Approach to Adaptive Problem Solving

  1996cited by this paper
• On reinforcement learning for robots

  1996cited by this paper
• Behavior coordination for a mobile robot using modular reinforcement learning

  1996cited by this paper
• Learning Control Composition in a Complex Environment

  1996cited by this paper
• Probabilistic Robot Navigation in Partially Observable Environments

  1995cited by this paper
• Learning from delayed rewards

  1995cited by this paper
• Quantitative Results Concerning the Utility of Explanation-Based Learning

  1995cited by this paper
• Improving Elevator Performance Using Reinforcement Learning

  1995cited by this paper
• TD Models: Modeling the World at a Mixture of Time Scales

  1995cited by this paper
• Temporal Difference Learning and TD-Gammon

  1995cited by this paper
• Learning to Act Using Real-Time Dynamic Programming

  1995cited by this paper
• Decomposition Techniques for Planning in Stochastic Domains

  1995cited by this paper
• Learning to Plan in Continuous Domains

  1994cited by this paper
• A Statistical Approach to Adaptive Problem-Solving for Large-Scale Scheduling and Resource Allocation Problems

  1994cited by this paper
• Robot Shaping: Developing Autonomous Agents Through Learning

  1994cited by this paper
• On the Convergence of Stochastic Iterative Dynamic Programming Algorithms

  1994cited by this paper
• Finding Structure in Reinforcement Learning

  1994cited by this paper
• Reasoning About Probabilistic Actions At Multiple Levels of Granularity

  1994cited by this paper
• Reinforcement Learning Methods for Continuous-Time Markov Decision Problems

  1994cited by this paper
• Hybrid Models for Motion Control Systems

  1993cited by this paper
• Hierarchical Learning in Stochastic Domains: Preliminary Results

  1993cited by this paper
• Teleo-Reactive Programs for Agent Control

  1993cited by this paper
• Learning and Executing Generalized Robot Plans

  1993cited by this paper
• Improving Generalization for Temporal Difference Learning: The Successor Representation

  1993cited by this paper
• Planning under Time Constraints in Stochastic Domains

  1993cited by this paper
• Robust Reinforcement Learning in Motion Planning

  1993cited by this paper
• Reinforcement Learning for Robots Using Neural Networks

  1992cited by this paper
• Feudal Reinforcement Learning

  1992cited by this paper
• The Ecient Learning of Multiple Task Sequences

  1992cited by this paper
• A Statistical Approach to Solving the EBL Utility Problem

  1992cited by this paper
• Reinforcement Learning with a Hierarchy of Abstract Models

  1992cited by this paper
• Automatic Programming of Robots Using Genetic Programming

  1992cited by this paper
• Made-Up Minds: A Constructivist Approach to Artificial Intelligence

  1991cited by this paper
• Scaling Reinforcement Learning Techniques via Modularity

  1991cited by this paper
• Automatic Programming of Behavior-Based Robots Using Reinforcement Learning

  1991cited by this paper
• Neural sequence chunkers

  1991cited by this paper
• The Efficient Learning of Multiple Task Sequences

  1991cited by this paper
• Why PRODIGY/EBL Works

  1990cited by this paper
• Learning to Coordinate Behaviors

  1990cited by this paper
• Learning Search Control Knowledge: An Explanation-Based Approach

  1988cited by this paper
• Quantitative Results Concerning the Utility of Explanation-based Learning

  1988cited by this paper
• Chunking in Soar

  1986cited by this paper
• A Qualitative Physics Based on Confluences

  1984cited by this paper
• Learning to solve problems by searching for macro-operators

  1983cited by this paper
• Common-Sense Knowledge of Space: Learning from Experience

  1979cited by this paper
• Planning in a Hierarchy of Abstraction Spaces

  1974cited by this paper
• Human Problem Solving.

  1973cited by this paper
• Dynamic Programming and Markov Processes

  1960cited by this paper

• In-Context Planning with Latent Temporal Abstractions

  2026cites this paper
• PaperGuide: Making Small Language-Model Paper-Reading Agents More Efficient

  2026influential citation
• Unsupervised Hierarchical Skill Discovery

  2026cites this paper
• Compositional Planning with Jumpy World Models

  2026influential citation
• Computability of Agentic Systems

  2026cites this paper
• Group-Invariant Unsupervised Skill Discovery: Symmetry-aware Skill Representations for Generalizable Behavior

  2026cites this paper
• Heterogeneous Multi-Expert Reinforcement Learning for Long-Horizon Multi-Goal Tasks in Autonomous Forklifts

  2026cites this paper
• Learning-Based Multi-Robot Active SLAM: A Conceptual Framework and Survey

  2026cites this paper
• Beyond Optimization: Intelligence as Metric-Topology Factorization under Geometric Incompleteness

  2026cites this paper
• MIRA: Memory-Integrated Reinforcement Learning Agent with Limited LLM Guidance

  2026cites this paper
• Expanding LLM Agent Boundaries with Strategy-Guided Exploration

  2026cites this paper
• Enabling Option Learning in Sparse Rewards with Hindsight Experience Replay

  2026cites this paper
• Intrinsic Stability Limits of Autoregressive Reasoning: Structural Consequences for Long-Horizon Execution

  2026influential citation
• APC-RL: Exceeding Data-Driven Behavior Priors with Adaptive Policy Composition

  2026cites this paper
• Understanding Symbols: Towards a Compact Vocabulary Learned from Robot Experience for Task Planning

  2026cites this paper
• A Review of Online Diffusion Policy RL Algorithms for Scalable Robotic Control

  2026cites this paper
• Policy-Based Deep Reinforcement Learning Hyperheuristics for Job-Shop Scheduling Problems

  2026cites this paper
• SIA: Symbolic Interpretability for Anticipatory Deep Reinforcement Learning in Network Control

  2026cites this paper
• Giving AI agents a sense of control facilitates reinforcement learning in multitasking scenarios

  2026cites this paper
• VGAS: Value-Guided Action-Chunk Selection for Few-Shot Vision-Language-Action Adaptation

  2026cites this paper
• Efficient Learning of Predictive Maps for Flexible Planning

  2026cites this paper
• Specialized Deep Residual Policy Reinforcement Learning Framework for Safe and Adaptive Continuous Control

  2026cites this paper
• Sparse and intelligent command of unmanned swarms via hierarchical multiagent reinforcement learning

  2026cites this paper
• Intermittent Active Inference

  2026cites this paper
• The Price Is Not Right: Neuro-Symbolic Methods Outperform VLAs on Structured Long-Horizon Manipulation Tasks with Significantly Lower Energy Consumption

  2026cites this paper
• A Scalable Approach to Solving Simulation-Based Network Security Games

  2026cites this paper
• Learning to Configure Agentic AI Systems

  2026cites this paper
• Robust robotic assembly via hierarchical diffusion policy-guided reinforcement learning

  2026cites this paper
• Joint Learning of Hierarchical Neural Options and Abstract World Model

  2026cites this paper
• A Modular Reinforcement Learning Framework for Iterative FPS Agent Development

  2026cites this paper
• Multiobjective Hierarchical Reinforcement Learning for Plant-Wide Optimization of Bayer Process

  2026cites this paper
• Multi-scale reinforcement learning framework for development policy optimization: Evidence from energy poverty alleviation

  2026cites this paper
• The Two-Stage Decision-Sampling Hypothesis: Understanding the Emergence of Self-Reflection in RL-Trained LLMs

  2026cites this paper
• Online Action-Stacking Improves Reinforcement Learning Performance for Air Traffic Control

  2026cites this paper
• Formal Methods in Robot Policy Learning and Verification: A Survey on Current Techniques and Future Directions

  2026cites this paper
• CTHA: Constrained Temporal Hierarchical Architecture for Stable Multi-Agent LLM Systems

  2026cites this paper
• Learning Transferable Skills in Action RPGs via Directed Skill Graphs and Selective Adaptation

  2026cites this paper
• Intrinsic Reward Policy Optimization for Sparse-Reward Environments

  2026influential citation
• Learning Abstractions for Hierarchical Planning in Program-Synthesis Agents

  2026cites this paper
• Enforcing Monotonic Progress in Legal Cross-Examination: Preventing Long-Horizon Stagnation in LLM-Based Inquiry

  2026cites this paper
• Semantically Labelled Automata for Multi-Task Reinforcement Learning with LTL Instructions

  2026cites this paper
• Object-Oriented Transition Modeling with Inductive Logic Programming

  2026cites this paper
• Intelligent AI Delegation

  2026cites this paper
• Hierarchical Successor Representation for Robust Transfer

  2026cites this paper
• Deep reinforcement learning-based multi-agent negotiation framework for conflict-free U-Space scenarios

  2026cites this paper
• Phase-Aware Mixture of Experts for Agentic Reinforcement Learning

  2026cites this paper
• Hierarchical Reward Design from Language: Enhancing Alignment of Agent Behavior with Human Specifications

  2026cites this paper
• Dynamic Swarm Reconfiguration via Multi-Level Adaptive Cooperative Architecture for Multi-Robot Exploration

  2026cites this paper
• HiMAC: Hierarchical Macro-Micro Learning for Long-Horizon LLM Agents

  2026cites this paper
• HiMAP-Travel: Hierarchical Multi-Agent Planning for Long-Horizon Constrained Travel

  2026cites this paper
• Framework for hierarchical deep reinforcement learning with conceptual embedding.

  2026cites this paper
• SoK: Agentic Skills -- Beyond Tool Use in LLM Agents

  2026influential citation
• Beyond Mimicry: Toward Lifelong Adaptability in Imitation Learning

  2026cites this paper
• HiPER: Hierarchical Reinforcement Learning with Explicit Credit Assignment for Large Language Model Agents

  2026cites this paper
• SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks

  2026cites this paper
• Gaia2: Benchmarking LLM Agents on Dynamic and Asynchronous Environments

  2026cites this paper
• Synthesis of model predictive control and reinforcement learning: Survey and classification

  2026cites this paper
• The Laplacian Keyboard: Beyond the Linear Span

  2026cites this paper
• The hidden risks of temporal resampling in clinical reinforcement learning

  2026cites this paper
• CADENT: Gated Hybrid Distillation for Sample-Efficient Transfer in Reinforcement Learning

  2026cites this paper
• Offline Discovery of Interpretable Skills from Multi-Task Trajectories

  2026cites this paper
• World Models as an Intermediary between Agents and the Real World

  2026cites this paper
• Scheduling Model Transmission and Training for Contact-Based Decentralized Satellite Federated Learning

  2026cites this paper
• Hierarchical Reinforcement Learning-Based Beam Selection for Integrated Sensing and Communication Systems

  2026cites this paper
• Semi-Markov Options Enabled DDPG Method for Autonomous Vehicle Overtaking With LiDAR and RADAR Fusion

  2026cites this paper
• Multi-Agent Hierarchical Fuzzy Reinforcement Learning for Cooperative-Competitive Peer-to-Peer Energy Trading With Privacy Preservation

  2026cites this paper
• Constrained Semi-MDP Formulation and Perception-Enhanced Safe Policy Learning for Efficient Dynamic Task Scheduling of Data Centers

  2026cites this paper
• VNF-FG Placement and Admission Control in SDN and NFV-Enabled IoT Networks: A Hierarchical Deep Reinforcement Learning Method

  2026cites this paper
• Hierarchical multi-agent reinforcement learning algorithm for multi-UAV roundup strategy

  2026cites this paper
• Evolving Programmatic Skill Networks

  2026cites this paper
• AtomMem : Learnable Dynamic Agentic Memory with Atomic Memory Operation

  2026cites this paper
• Personalized skill transfer optimization in swimming training through multi-agent reinforcement learning driven digital twin environments

  2026cites this paper
• Subgoal-Based Hierarchical Reinforcement Learning for Multiagent Collaboration

  2026cites this paper
• Hierarchical Brain-Inspired Deep Learning for Autonomous Decision-Making in Complex Dynamic Environments

  2026cites this paper
• Human intergroup coordination in a hierarchical multi-agent sensorimotor task arises from concurrent co-optimization

  2025cites this paper
• Hierarchical Reinforcement Learning in Multi-Goal Spatial Navigation with Autonomous Mobile Robots

  2025influential citation
• Cognitive maps are generative programs

  2025cites this paper
• Deep Reinforcement Learning Based Navigation with Macro Actions and Topological Maps

  2025influential citation
• Wake Homing Torpedo Guidance Using a Hierarchical Deep Reinforcement Learning Framework

  2025cites this paper
• Simultaneous design of reconfigurable manufacturing systems and their production plans using hierarchical reinforcement learning

  2025cites this paper
• An event-triggered deep reinforcement learning guidance algorithm for intercepting maneuvering target of speed advantage.

  2025cites this paper
• Efficient Policy Iteration and Value Iteration via State Space Clustering

  2025cites this paper
• Modular electric units for first-and-last-mile reservation services considering uncertainty

  2025cites this paper
• Skill Learning Using a Single Demonstration for Hierarchical Robotic Navigation

  2025cites this paper
• Exploring Inevitable Waypoints for Unsolvability Explanation in Hybrid Planning Problems

  2025cites this paper
• Towards a No Code Deployment of Social Robotics Use Cases

  2025cites this paper
• MOSAIC: A Skill-Centric Algorithmic Framework for Long-Horizon Manipulation Planning

  2025cites this paper
• Nearly Lossless Zero Shot Learning for Robot Navigation Using Declarative Re-Planning

  2025cites this paper
• Constructing an Optimal Behavior Basis for the Option Keyboard

  2025cites this paper
• Deep deterministic policy gradient for volt-var control

  2025cites this paper
• Reinforcement Learning for Fail-Operational Systems with Disentangled Dual-Skill Variables

  2025cites this paper
• Hierarchical reinforcement learning based on macro actions

  2025cites this paper
• Multi-Length Skills with Priors for Reinforcement Learning

  2025cites this paper
• TW-CRL: Time-Weighted Contrastive Reward Learning for Efficient Inverse Reinforcement Learning

  2025cites this paper
• Enhanced hierarchical reinforcement learning for co-optimization of HVAC system operations

  2025cites this paper
• Hierarchical reinforcement learning with curriculum demonstrations and goal-guided policies for sequential robotic manipulation

  2025cites this paper
• A Classification View on Meta Learning Bandits

  2025cites this paper
• Illustrated Landmark Graphs for Long-horizon Policy Learning

  2025cites this paper
• Dexterous Manipulation Through Imitation Learning: A Survey

  2025cites this paper
• Solving Sokoban using Hierarchical Reinforcement Learning with Landmarks

  2025cites this paper