A brain-computer interface for high-level remote control of an autonomous, reinforcement-learning-based robotic system for reaching and grasping

We present an Internet-based brain-computer interface (BCI) for controlling an intelligent robotic device with autonomous reinforcement-learning. BCI control was achieved through dry-electrode electroencephalography (EEG) obtained during imaginary movements. Rather than using low-level direct motor control, we employed a high-level control scheme of the robot, acquired via reinforcement learning, to keep the users cognitive load low while allowing control a reaching-grasping task with multiple degrees of freedom. High-level commands were obtained by classification of EEG responses using an artificial neural network approach utilizing time-frequency features and conveyed through an intuitive user interface. The novel ombination of a rapidly operational dry electrode setup, autonomous control and Internet connectivity made it possible to conveniently interface subjects in an EEG laboratory with remote robotic devices in a closed-loop setup with online visual feedback of the robots actions to the subject. The same approach is also suitable to provide home-bound patients with the possibility to control state-of-the-art robotic devices currently confined to a research environment. Thereby, our BCI approach could help severely paralyzed patients by facilitating patient-centered research of new means of communication, mobility and independence.

• Publication year

  2014

• Venue

  International Conference on Intelligent User Interfaces

• Publication date

  2014-02-24

• Fields of study

  Computer Science, Engineering

• Identifiers
  DOI 10.1145/2557500.2557533
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Acquiring visual servoing reaching and grasping skills using neural reinforcement learning

  2013cited by this paper
• High-performance neuroprosthetic control by an individual with tetraplegia.

  2013cited by this paper
• Multi-Class Motor Imagery EEG Decoding for Brain-Computer Interfaces

  2012cited by this paper
• Toward smarter BCIs: extending BCIs through hybridization and intelligent control

  2012cited by this paper
• Reach and grasp by people with tetraplegia using a neurally controlled robotic arm

  2012cited by this paper
• Toward Unsupervised Adaptation of LDA for Brain–Computer Interfaces

  2011cited by this paper
• Machine-Learning-Based Coadaptive Calibration for Brain-Computer Interfaces

  2011cited by this paper
• An adaptive brain-computer interface for humanoid robot control

  2011cited by this paper
• Evolution of brain-computer interface: action potentials, local field potentials and electrocorticograms.

  2010cited by this paper
• Human brain-teleoperated robot between remote places

  2009cited by this paper
• A review on directional information in neural signals for brain-machine interfaces.

  2009cited by this paper
• Spatial filtering and selection of optimized components in four class motor imagery EEG data using independent components analysis

  2007cited by this paper
• A review of classification algorithms for EEG-based brain–computer interfaces

  2007cited by this paper
• Motor imagery and EEG-based control of spelling devices and neuroprostheses.

  2006cited by this paper
• Teleoperation of an Industrial Manipulator Through a TCP/IP Channel Using EEG Signals

  2006cited by this paper
• Neural Fitted Q Iteration - First Experiences with a Data Efficient Neural Reinforcement Learning Method

  2005cited by this paper
• BCI2000: a general-purpose brain-computer interface (BCI) system

  2004cited by this paper
• Cognitive Control Signals for Neural Prosthetics

  2004cited by this paper
• The five percent electrode system for high-resolution EEG and ERP measurements.

  2001cited by this paper
• Event-related EEG/MEG synchronization and desynchronization: basic principles.

  1999cited by this paper
• Introduction to Reinforcement Learning

  1998cited by this paper
• A Practical Bayesian Framework for Backpropagation Networks

  1992cited by this paper
• Spectrum estimation and harmonic analysis

  1982cited by this paper

• Brain-controlled operator model-driven deep reinforcement learning for adaptive brain-machine collaborative control

  2026cites this paper
• An integration of BCI and vision system for Robotics applications: a review

  2025cites this paper
• DEER: A Delay-Resilient Framework for Reinforcement Learning with Variable Delays

  2024cites this paper
• Hierarchical Procedural Framework for Low-latency Robot-Assisted Hand-Object Interaction

  2024cites this paper
• Real-Time Dynamic Robot-Assisted Hand-Object Interaction via Motion Primitives

  2024cites this paper
• A Conversational Brain-Artificial Intelligence Interface

  2024cites this paper
• The Experimentalist’s Guide to Machine Learning for Small Molecule Design

  2023cites this paper
• RLSF: Multimodal Sleep Improvement Based Reinforcement Learning

  2023cites this paper
• Bootstrapping Human-Autonomy Collaborations by using Brain-Computer Interface of SSVEP for Multi-Agent Deep Reinforcement Learning

  2022cites this paper
• Understanding HCI Practices and Challenges of Experiment Reporting with Brain Signals: Towards Reproducibility and Reuse

  2022cites this paper
• Improving HCI with Brain Input: Review, Trends, and Outlook

  2021cites this paper
• A review on Virtual Reality and Augmented Reality use-cases of Brain Computer Interface based applications for smart cities

  2021cites this paper
• Using Brain-Computer Interface to Control a Virtual Drone Using Non-Invasive Motor Imagery and Machine Learning

  2021cites this paper
• A closed-loop brain-computer interface with augmented reality feedback for industrial human-robot collaboration

  2021cites this paper
• MIndGrasp: A New Training and Testing Framework for Motor Imagery Based 3-Dimensional Assistive Robotic Control

  2020cites this paper
• An online SSVEP-BCI system in an optical see-through augmented reality environment

  2020cites this paper
• Development of Smartphone-Based Human-Robot Interfaces for Individuals With Disabilities

  2020cites this paper
• Combination of Augmented Reality Based Brain- Computer Interface and Computer Vision for High-Level Control of a Robotic Arm

  2020cites this paper
• A Brain-Computer Interface for arbitrary Pick and Place Tasks

  2019cites this paper
• Intuitive Adaptive Orientation Control for Enhanced Human–Robot Interaction

  2019cites this paper
• Autonomy infused teleoperation with application to brain computer interface controlled manipulation

  2017cites this paper
• Kinova Modular Robot Arms for Service Robotics Applications

  2017cites this paper
• Kinova Modular Robot Arms for Service Robotics Applications

  2017cites this paper
• Robotic arm control using hybrid brain-machine interface and augmented reality feedback

  2017cites this paper
• Brain-Computer Interfaces and Augmented Reality: a State of the Art

  2017cites this paper
• Applications of Deep Learning and Reinforcement Learning to Biological Data

  2017cites this paper
• Décodage des intentions et des exécutions motrices : étude du rôle des oscillations cérébrales via l’apprentissage machine et développement d’outils open-source

  2017cites this paper
• Closed-Loop Hybrid Gaze Brain-Machine Interface Based Robotic Arm Control with Augmented Reality Feedback

  2017cites this paper
• Why Cannot Control Your Smartphones by Thinking? Hands-Free Information Control System Based on EEG

  2016cites this paper
• An autonomous robotic assistant for drinking

  2015cites this paper
• Contribution to the study of the use of brain-computer interfaces in virtual and augmented reality

  2015cites this paper
• Grasping with Your Brain: A Brain-Computer Interface for Fast Grasp Selection

  2015cites this paper
• Contribution to the study of the use of brain-computer interfaces in virtual and augmented reality. (Contribution à l'étude de l'utilisation des interfaces cerveau-ordinateur en réalité virtuelle ou augmentée)

  2015cites this paper
• Hybrid gaze/EEG brain computer interface for robot arm control on a pick and place task

  2015cites this paper
• User-centered gesture development in TV viewing environment

  2014cites this paper
• Journal of Neuroscience Methods

  year unknowncites this paper