Focused model-learning and planning for non-Gaussian continuous state-action systems

We introduce a framework for model learning and planning in stochastic domains with continuous state and action spaces and non-Gaussian transition models. It is efficient because (1) local models are estimated only when the planner requires them; (2) the planner focuses on the most relevant states to the current planning problem; and (3) the planner focuses on the most informative and/or high-value actions. Our theoretical analysis shows the validity and asymptotic optimality of the proposed approach. Empirically, we demonstrate the effectiveness of our algorithm on a simulated multi-modal pushing problem.

• Publication year

  2016

• Venue

  IEEE International Conference on Robotics and Automation

• Publication date

  2016-07-26

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1109/ICRA.2017.7989433arXiv 1607.07762
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• D Analyse

  2019cited by this paper
• More than a million ways to be pushed. A high-fidelity experimental dataset of planar pushing

  2016cited by this paper
• Safe Exploration in Finite Markov Decision Processes with Gaussian Processes

  2016cited by this paper
• Monte Carlo Tree Search in Continuous Action Spaces with Execution Uncertainty

  2016cited by this paper
• Optimism-driven exploration for nonlinear systems

  2015cited by this paper
• Bayesian Reinforcement Learning: A Survey

  2015cited by this paper
• Optimization as Estimation with Gaussian Processes in Bandit Settings

  2015cited by this paper
• Continuous Action-Space Reinforcement Learning Methods Applied to the Minimum-Time Swing-Up of the Acrobot

  2015cited by this paper
• An incremental sampling-based algorithm for stochastic optimal control

  2012influential reference
• Reinforcement learning with guided policy search using Gaussian processes

  2012cited by this paper
• Bandit-Based Planning and Learning in Continuous-Action Markov Decision Processes

  2012cited by this paper
• Consistency of the kernel density estimator: a survey

  2012cited by this paper
• Model learning for robot control: a survey

  2011cited by this paper
• PILCO: A Model-Based and Data-Efficient Approach to Policy Search

  2011cited by this paper
• Learning to predict how rigid objects behave under simple manipulation

  2011cited by this paper
• Variational Heteroscedastic Gaussian Process Regression

  2011cited by this paper
• Sample-Based Planning for Continuous Action Markov Decision Processes

  2011cited by this paper
• A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning

  2010cited by this paper
• Prediction learning in robotic manipulation

  2010cited by this paper
• Settling the Polynomial Learnability of Mixtures of Gaussians

  2010cited by this paper
• Adaptive autonomous control using online value iteration with gaussian processes

  2009cited by this paper
• Gaussian process dynamic programming

  2009cited by this paper
• Prediction learning in robotic pushing manipulation

  2009cited by this paper
• Variational Mixture of Gaussian Process Experts

  2008cited by this paper
• Model-Based Reinforcement Learning with Continuous States and Actions

  2008cited by this paper
• Using Gaussian Processes to Optimize Expensive Functions

  2008cited by this paper
• Reinforcement Learning in Continuous Action Spaces

  2007cited by this paper
• The Stochastic Motion Roadmap: A Sampling Framework for Planning with Markov Motion Uncertainty

  2007cited by this paper
• Focused Real-Time Dynamic Programming for MDPs: Squeezing More Out of a Heuristic

  2006cited by this paper
• Planning algorithms

  2006influential reference
• Bounded real-time dynamic programming: RTDP with monotone upper bounds and performance guarantees

  2005cited by this paper
• Cours d'analyse de l'école royale polytechnique

  2004cited by this paper
• Gaussian Processes in Machine Learning

  2003cited by this paper
• Randomized Kinodynamic Planning

  1999cited by this paper
• Reinforcement Learning: An Introduction

  1998cited by this paper
• Rapidly-exploring random trees : a new tool for path planning

  1998cited by this paper
• Learning to Act Using Real-Time Dynamic Programming

  1995cited by this paper
• Acquiring robot skills via reinforcement learning

  1994cited by this paper
• Reinforcement Learning With High-Dimensional, Continuous Actions

  1993cited by this paper
• On the efficiency of certain quasi-random sequences of points in evaluating multi-dimensional integrals

  1960cited by this paper

• QuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?

  2025cites this paper
• Grammar Prompting for Domain-Specific Language Generation with Large Language Models

  2023cites this paper
• Active and Compact Entropy Search for High-Dimensional Bayesian Optimization

  2023cites this paper
• HyperBO+: Pre-training a universal prior for Bayesian optimization with hierarchical Gaussian processes

  2022cites this paper
• Pre-training helps Bayesian optimization too

  2022cites this paper
• Bad priors, their threats to Bayesian optimization and some remedies via prior learning

  2022cites this paper
• Pre-trained Gaussian Processes for Bayesian Optimization

  2021cites this paper
• Transporting a heavy object on a frictional floor by a mobile manipulator based on adaptive MPC framework

  2021cites this paper
• Neural fidelity warping for efficient robot morphology design

  2020cites this paper
• Batch Bayesian optimization via adaptive local search

  2020cites this paper
• Learning Manipulation States and Actions for Efficient Non-prehensile Rearrangement Planning

  2019cites this paper
• Online Learning in Planar Pushing with Combined Prediction Model

  2019cites this paper
• Learning structured transition models for multi-object manipulation

  2018cites this paper
• Push-Net: Deep Planar Pushing for Objects with Unknown Physical Properties

  2018cites this paper
• Maximizing acquisition functions for Bayesian optimization

  2018cites this paper
• Max-value Entropy Search for Efficient Bayesian Optimization

  2017cites this paper
• Batched High-dimensional Bayesian Optimization via Structural Kernel Learning

  2017cites this paper