Learning Kinematic Descriptions using SPARE: Simulated and Physical ARticulated Extendable dataset

Next generation robots will need to understand intricate and articulated objects as they cooperate in human environments. To do so, these robots will need to move beyond their current abilities--- working with relatively simple objects in a task-indifferent manner--- toward more sophisticated abilities that dynamically estimate the properties of complex, articulated objects. To that end, we make two compelling contributions toward general articulated (physical) object understanding in this paper. First, we introduce a new dataset, SPARE: Simulated and Physical ARticulated Extendable dataset. SPARE is an extendable open-source dataset providing equivalent simulated and physical instances of articulated objects (kinematic chains), providing the greater research community with a training and evaluation tool for methods generating kinematic descriptions of articulated objects. To the best of our knowledge, this is the first joint visual and physical (3D-printable) dataset for the Vision community. Second, we present a deep neural network that can predit the number of links and the length of the links of an articulated object. These new ideas outperform classical approaches to understanding kinematic chains, such tracking-based methods, which fail in the case of occlusion and do not leverage multiple views when available.

• Publication year

  2018

• Venue

  arXiv.org

• Publication date

  2018-03-29

• Fields of study

  Computer Science, Engineering

• Identifiers
  arXiv 1803.11147
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Domain randomization for transferring deep neural networks from simulation to the real world

  2017cited by this paper
• Feature selection for learning versatile manipulation skills based on observed and desired trajectories

  2017cited by this paper
• Connecting Look and Feel: Associating the Visual and Tactile Properties of Physical Materials

  2017cited by this paper
• Learning a visuomotor controller for real world robotic grasping using simulated depth images

  2017cited by this paper
• Virtual vs. real: Trading off simulations and physical experiments in reinforcement learning with Bayesian optimization

  2017cited by this paper
• RGB-D datasets using microsoft kinect or similar sensors: a survey

  2016cited by this paper
• Deep Kinematic Pose Regression

  2016cited by this paper
• Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Model

  2016cited by this paper
• Sim-to-Real Robot Learning from Pixels with Progressive Nets

  2016cited by this paper
• Spatiotemporal Articulated Models for Dynamic SLAM

  2016cited by this paper
• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation

  2016cited by this paper
• Deep Feature Learning for Acoustics-Based Terrain Classification

  2015cited by this paper
• Galileo: Perceiving Physical Object Properties by Integrating a Physics Engine with Deep Learning

  2015cited by this paper
• ActivityNet: A large-scale video benchmark for human activity understanding

  2015cited by this paper
• End-to-End Training of Deep Visuomotor Policies

  2015cited by this paper
• Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments

  2014cited by this paper
• Real-Time Simultaneous Pose and Shape Estimation for Articulated Objects Using a Single Depth Camera

  2014cited by this paper
• 3D ShapeNets: A deep representation for volumetric shapes

  2014cited by this paper
• ImageNet Large Scale Visual Recognition Challenge

  2014cited by this paper
• Pose Machines: Articulated Pose Estimation via Inference Machines

  2014cited by this paper
• Learning Articulated Motions From Visual Demonstration

  2014cited by this paper
• Efficient Hierarchical Graph-Based Segmentation of RGBD Videos

  2014cited by this paper
• Large-Scale Video Classification with Convolutional Neural Networks

  2014cited by this paper
• Interactive segmentation, tracking, and kinematic modeling of unknown 3D articulated objects

  2013cited by this paper
• MODEC: Multimodal Decomposable Models for Human Pose Estimation

  2013cited by this paper
• Strong Appearance and Expressive Spatial Models for Human Pose Estimation

  2013cited by this paper
• Clustered Pose and Nonlinear Appearance Models for Human Pose Estimation

  2010cited by this paper
• Active perception and scene modeling by planning with probabilistic 6D object poses

  2010cited by this paper
• A Factorization-Based Approach for Articulated Nonrigid Shape, Motion and Kinematic Chain Recovery From Video

  2008cited by this paper
• Design and use paradigms for Gazebo, an open-source multi-robot simulator

  2004cited by this paper
• 7 – Geometric and Neural Models of Object Perception

  2001cited by this paper
• From fragments to objects : segmentation and grouping in vision

  2001cited by this paper
• Long Short-Term Memory

  1997cited by this paper
• Constraints on perceptual learning: objects and dimensions.

  1995cited by this paper

• Controlled Agentic Planning & Reasoning for Mechanism Synthesis

  2025cites this paper
• Estimating Mass Distribution of Articulated Objects using Non-prehensile Manipulation

  2020cites this paper
• Estimating Mass Distribution of Articulated Objects through Physical Interaction

  2019cites this paper
• Estimating Mass Distribution of Articulated Objects through Non-prehensile Manipulation

  2019cites this paper