Modeling Grasp Motor Imagery Through Deep Conditional Generative Models

Grasping is a complex process involving knowledge of the object, the surroundings, and of oneself. While humans are able to integrate and process all of the sensory information required for performing this task, equipping machines with this capability is an extremely challenging endeavor. In this paper, we investigate how deep learning techniques can allow us to translate high-level concepts such as motor imagery to the problem of robotic grasp synthesis. We explore a paradigm based on generative models for learning integrated object-action representations and demonstrate its capacity for capturing and generating multimodal multifinger grasp configurations on a simulated grasping dataset.

• Publication year

  2017

• Venue

  IEEE Robotics and Automation Letters

• Publication date

  2017-01-11

• Fields of study

  Mathematics, Computer Science, Engineering

• Identifiers
  DOI 10.1109/LRA.2017.2651945arXiv 1701.03041
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks

  2016cited by this paper
• Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection

  2016cited by this paper
• Dex-Net 1.0: A cloud-based network of 3D objects for robust grasp planning using a Multi-Armed Bandit model with correlated rewards

  2016cited by this paper
• Deep spatial autoencoders for visuomotor learning

  2015cited by this paper
• Deep multimodal embedding: Manipulating novel objects with point-clouds, language and trajectories

  2015cited by this paper
• Domain-Adversarial Training of Neural Networks

  2015cited by this paper
• Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours

  2015cited by this paper
• End-to-End Training of Deep Visuomotor Policies

  2015cited by this paper
• Leveraging big data for grasp planning

  2015cited by this paper
• Learning Structured Output Representation using Deep Conditional Generative Models

  2015cited by this paper
• The effect of somatosensory input on motor imagery depends upon motor imagery capability

  2015cited by this paper
• Multimodal deep autoencoders for control of a mobile robot

  2015cited by this paper
• G3DB: A database of successful and failed grasps with RGB-D images, point clouds, mesh models and gripper parameters

  2015cited by this paper
• Lasagne: First release.

  2015cited by this paper
• Stochastic Backpropagation and Approximate Inference in Deep Generative Models

  2014cited by this paper
• Multimodal integration learning of robot behavior using deep neural networks

  2014cited by this paper
• V-REP: A versatile and scalable robot simulation framework

  2013cited by this paper
• Network In Network

  2013cited by this paper
• Auto-Encoding Variational Bayes

  2013cited by this paper
• Grasp Moduli Spaces

  2013cited by this paper
• Does motor imagery share neural networks with executed movement: a multivariate fMRI analysis

  2013cited by this paper
• Deep learning for detecting robotic grasps

  2013cited by this paper
• Multivariate discretization for Bayesian Network structure learning in robot grasping

  2011cited by this paper
• Cognitive motor processes: the role of motor imagery in the study of motor representations.

  2009cited by this paper
• Motor planning, imagery, and execution in the distributed motor network: a time-course study with functional MRI.

  2008cited by this paper
• Learning Object Affordances: From Sensory--Motor Coordination to Imitation

  2008cited by this paper
• Visualizing Data using t-SNE

  2008cited by this paper
• Motor imagery and action observation: cognitive tools for rehabilitation

  2007cited by this paper
• Neural Simulation of Action: A Unifying Mechanism for Motor Cognition

  2001cited by this paper
• Orientation of the opposition axis in mentally simulated grasping

  2000cited by this paper
• Integration of Visual and Somatosensory Information for Preshaping Hand in Grasping Movements

  1992cited by this paper

• DexGANGrasp: Dexterous Generative Adversarial Grasping Synthesis for Task-Oriented Manipulation

  2024cites this paper
• Pick and Place Planning is Better Than Pick Planning Then Place Planning

  2024cites this paper
• Transferring Grasping Across Grippers: Learning–Optimization Hybrid Framework for Generalized Planar Grasp Generation

  2024cites this paper
• Multivariate Fast Iterative Filtering Based Automated System for Grasp Motor Imagery Identification Using EEG Signals

  2023cites this paper
• Probabilistic Pick and Place Planning instead of Pick then Place Planning

  2023cites this paper
• Planning Visual-Tactile Precision Grasps via Complementary Use of Vision and Touch

  2022cites this paper
• Faster than Real-Time Surface Pose Estimation with Application to Autonomous Robotic Grasping

  2022cites this paper
• DcnnGrasp: towards accurate grasp pattern recognition with adaptive regularizer learning

  2022cites this paper
• Learning to Reorient Objects With Stable Placements Afforded by Extrinsic Supports

  2022cites this paper
• Deep Learning Approaches to Grasp Synthesis: A Review

  2022cites this paper
• FFHNet: Generating Multi-Fingered Robotic Grasps for Unknown Objects in Real-time

  2022cites this paper
• GKNet: Grasp keypoint network for grasp candidates detection

  2021cites this paper
• NeuroGrasp: Real-Time EEG Classification of High-Level Motor Imagery Tasks Using a Dual-Stage Deep Learning Framework

  2021cites this paper
• Action Image Representation: Learning Scalable Deep Grasping Policies with Zero Real World Data

  2020cites this paper
• Multi-Fingered Active Grasp Learning

  2020cites this paper
• Multifingered Grasp Planning via Inference in Deep Neural Networks: Outperforming Sampling by Learning Differentiable Models

  2020influential citation
• Incorporating Object Intrinsic Features Within Deep Grasp Affordance Prediction

  2020cites this paper
• AdaGrasp: Learning an Adaptive Gripper-Aware Grasping Policy

  2020cites this paper
• Deep Learning Method for Grasping Novel Objects Using Dexterous Hands

  2020influential citation
• Decoding of Grasp Motions from EEG Signals Based on a Novel Data Augmentation Strategy

  2020cites this paper
• Modeling Grasp Type Improves Learning-Based Grasp Planning

  2019cites this paper
• Deep Dexterous Grasping of Novel Objects from a Single View

  2019cites this paper
• Computing and Evaluating Adversarial Grasp Objects with Systematic Face Rotations and Applications of Dense Object Descriptors to Mechanical Search

  2019cites this paper
• Increasing the Generalisaton Capacity of Conditional VAEs

  2019cites this paper
• Learning Continuous 3D Reconstructions for Geometrically Aware Grasping

  2019cites this paper
• UniGrasp: Learning a Unified Model to Grasp with N-Fingered Robotic Hands

  2019cites this paper
• UniGrasp: Learning a Unified Model to Grasp With Multifingered Robotic Hands

  2019cites this paper
• Student Responses to RENEWAL Questions

  2019cites this paper
• Walking Imagery Evaluation in Brain Computer Interfaces via a Multi-View Multi-Level Deep Polynomial Network

  2019cites this paper
• Adversarial Grasp Objects

  2019cites this paper
• Learning better generative models for dexterous, single-view grasping of novel objects

  2019cites this paper
• Mind the Gap! Bridging the Reality Gap in Visual Perception and Robotic Grasping with Domain Randomisation

  2018cites this paper
• EnsembleNet: Improving Grasp Detection using an Ensemble of Convolutional Neural Networks

  2018cites this paper
• Learning deep representations for robotics applications

  2018cites this paper
• Planning Multi-Fingered Grasps as Probabilistic Inference in a Learned Deep Network

  2018cites this paper
• An Integrated Simulator and Dataset that Combines Grasping and Vision for Deep Learning

  2017cites this paper
• Parallel-Pathway Generator for Generative Adversarial Networks to Generate High-Resolution Natural Images

  2017cites this paper
• Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics

  2017cites this paper