A Survey on Deep Learning Methods for Robot Vision

Deep learning has allowed a paradigm shift in pattern recognition, from using hand-crafted features together with statistical classifiers to using general-purpose learning procedures for learning data-driven representations, features, and classifiers together. The application of this new paradigm has been particularly successful in computer vision, in which the development of deep learning methods for vision applications has become a hot research topic. Given that deep learning has already attracted the attention of the robot vision community, the main purpose of this survey is to address the use of deep learning in robot vision. To achieve this, a comprehensive overview of deep learning and its usage in computer vision is given, that includes a description of the most frequently used neural models and their main application areas. Then, the standard methodology and tools used for designing deep-learning based vision systems are presented. Afterwards, a review of the principal work using deep learning in robot vision is presented, as well as current and future trends related to the use of deep learning in robotics. This survey is intended to be a guide for the developers of robot vision systems.

• Publication year

  2018

• Venue

  arXiv.org

• Publication date

  2018-03-28

• Fields of study

  Computer Science, Engineering

• Identifiers
  arXiv 1803.10862
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Regularization of Neural Network Using DropCoadapt

  2020cited by this paper
• Places: A 10 Million Image Database for Scene Recognition

  2018cited by this paper
• Stacked Deconvolutional Network for Semantic Segmentation

  2017cited by this paper
• Understanding Convolution for Semantic Segmentation

  2017cited by this paper
• Learning Object Interactions and Descriptions for Semantic Image Segmentation

  2017influential reference
• Deep Dual Learning for Semantic Image Segmentation

  2017influential reference
• Scale-Adaptive Convolutions for Scene Parsing

  2017cited by this paper
• Using Convolutional Neural Networks in Robots with Limited Computational Resources: Detecting NAO Robots while Playing Soccer

  2017influential reference
• Adversarial PoseNet: A Structure-Aware Convolutional Network for Human Pose Estimation

  2017cited by this paper
• Residual Attention Network for Image Classification

  2017cited by this paper
• Rethinking Atrous Convolution for Semantic Image Segmentation

  2017influential reference
• Squeeze-and-Excitation Networks

  2017cited by this paper
• Exploring convolutional networks for end-to-end visual servoing

  2017cited by this paper
• Combining Semantic and Geometric Features for Object Class Segmentation of Indoor Scenes

  2017cited by this paper
• Accurate Single Stage Detector Using Recurrent Rolling Convolution

  2017cited by this paper
• Multi-stream deep networks for human action classification with sequential tensor decomposition

  2017influential reference
• Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors

  2016cited by this paper
• Stacked Hourglass Networks for Human Pose Estimation

  2016influential reference
• Deep learning for visual understanding: A review

  2016influential reference
• Dex-Net 1.0: A cloud-based network of 3D objects for robust grasp planning using a Multi-Armed Bandit model with correlated rewards

  2016cited by this paper
• RotationNet: Learning Object Classification Using Unsupervised Viewpoint Estimation

  2016cited by this paper
• A Taxonomy of Deep Convolutional Neural Nets for Computer Vision

  2016influential reference
• Object discovery and grasp detection with a shared convolutional neural network

  2016cited by this paper
• Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation

  2016cited by this paper
• XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks

  2016cited by this paper
• Robust single-view instance recognition

  2016cited by this paper
• Vehicle Detection from 3D Lidar Using Fully Convolutional Network

  2016influential reference
• Fusing LIDAR and images for pedestrian detection using convolutional neural networks

  2016cited by this paper
• Real-time 3D scene layout from a single image using Convolutional Neural Networks

  2016cited by this paper
• Deep learning for human part discovery in images

  2016cited by this paper
• EIE: Efficient Inference Engine on Compressed Deep Neural Network

  2016cited by this paper
• TSC-DL: Unsupervised trajectory segmentation of multi-modal surgical demonstrations with Deep Learning

  2016cited by this paper
• Mastering the game of Go with deep neural networks and tree search

  2016cited by this paper
• A Machine Learning Approach to Visual Perception of Forest Trails for Mobile Robots

  2016cited by this paper
• Detection of pedestrians at far distance

  2016cited by this paper
• Convolutional hypercube pyramid for accurate RGB-D object category and instance recognition

  2016cited by this paper
• Target-driven visual navigation in indoor scenes using deep reinforcement learning

  2016cited by this paper
• Scale-Adaptive Deconvolutional Regression Network for Pedestrian Detection

  2016cited by this paper
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

  2016cited by this paper
• Multi-Stream Multi-Class Fusion of Deep Networks for Video Classification

  2016cited by this paper
• RotationNet: Joint Object Categorization and Pose Estimation Using Multiviews from Unsupervised Viewpoints

  2016cited by this paper
• Pyramid Scene Parsing Network

  2016influential reference
• Exploit All the Layers: Fast and Accurate CNN Object Detector with Scale Dependent Pooling and Cascaded Rejection Classifiers

  2016cited by this paper
• Street-view change detection with deconvolutional networks

  2016influential reference
• Identity Mappings in Deep Residual Networks

  2016cited by this paper
• Object recognition using local invariant features for robotic applications: A survey

  2016influential reference
• Laplacian Reconstruction and Refinement for Semantic Segmentation

  2016cited by this paper
• Exploring Representation Learning With CNNs for Frame-to-Frame Ego-Motion Estimation

  2016cited by this paper
• R-FCN: Object Detection via Region-based Fully Convolutional Networks

  2016cited by this paper
• Wider or Deeper: Revisiting the ResNet Model for Visual Recognition

  2016cited by this paper
• Exploiting fully convolutional neural networks for fast road detection

  2016cited by this paper
• Subcategory-Aware Convolutional Neural Networks for Object Proposals and Detection

  2016cited by this paper
• Action Recognition Based on Efficient Deep Feature Learning in the Spatio-Temporal Domain

  2016cited by this paper
• Theano: A Python framework for fast computation of mathematical expressions

  2016cited by this paper
• A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detection

  2016cited by this paper
• Densely Connected Convolutional Networks

  2016cited by this paper
• Ball Localization for Robocup Soccer Using Convolutional Neural Networks

  2016cited by this paper
• Multi-modal Auto-Encoders as Joint Estimators for Robotics Scene Understanding

  2016cited by this paper
• Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning

  2016cited by this paper
• Deep metric learning autoencoder for nonlinear temporal alignment of human motion

  2016cited by this paper
• A robot exploration strategy based on Q-learning network

  2016cited by this paper
• Sample Efficient Actor-Critic with Experience Replay

  2016cited by this paper
• Feature Pyramid Networks for Object Detection

  2016cited by this paper
• Part-based room categorization for household service robots

  2016cited by this paper
• Robobarista: Learning to Manipulate Novel Objects via Deep Multimodal Embedding

  2016cited by this paper
• What can i do around here? Deep functional scene understanding for cognitive robots

  2016influential reference
• gvnn: Neural Network Library for Geometric Computer Vision

  2016cited by this paper
• Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection

  2016influential reference
• Crafting GBD-Net for Object Detection

  2016cited by this paper
• Beyond Skip Connections: Top-Down Modulation for Object Detection

  2016cited by this paper
• A Deep Learning Approach for Object Recognition with NAO Soccer Robots

  2016cited by this paper
• Learning to Navigate in Complex Environments

  2016cited by this paper
• Cross-modal adaptation for RGB-D detection

  2016cited by this paper
• Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and Reading Books

  2015cited by this paper
• Place categorization and semantic mapping on a mobile robot

  2015influential reference
• Understand scene categories by objects: A semantic regularized scene classifier using Convolutional Neural Networks

  2015influential reference
• Convolutional neural network-based image representation for visual loop closure detection

  2015cited by this paper
• Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding

  2015cited by this paper
• Deep Residual Learning for Image Recognition

  2015influential reference
• SINGA: Putting Deep Learning in the Hands of Multimedia Users

  2015cited by this paper
• Deep Convolutional Neural Networks for pedestrian detection

  2015influential reference
• Human-level control through deep reinforcement learning

  2015influential reference
• Object Detection: Current and Future Directions

  2015cited by this paper
• Multimodal deep learning for robust RGB-D object recognition

  2015cited by this paper
• Fast R-CNN

  2015influential reference
• SINGA: A Distributed Deep Learning Platform

  2015cited by this paper
• SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation

  2015influential reference
• Deep Learning

  2015influential reference
• Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours

  2015cited by this paper
• Training a Convolutional Neural Network for Appearance-Invariant Place Recognition

  2015cited by this paper
• Modelling uncertainty in deep learning for camera relocalization

  2015cited by this paper
• LSTM: A Search Space Odyssey

  2015cited by this paper
• Taking a deeper look at pedestrians

  2015cited by this paper
• U-Net: Convolutional Networks for Biomedical Image Segmentation

  2015cited by this paper
• Real-world Object Recognition with Off-the-shelf Deep Conv Nets: How Many Objects can iCub Learn?

  2015cited by this paper
• Deep spatial autoencoders for visuomotor learning

  2015influential reference
• Chainer : a Next-Generation Open Source Framework for Deep Learning

  2015cited by this paper
• Comparative Study of Deep Learning Software Frameworks

  2015cited by this paper
• Beyond short snippets: Deep networks for video classification

  2015cited by this paper
• Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

  2015cited by this paper

• Behavior Detection of Quadruped Companion Robots Using CNN: Towards Closer Human-Robot Cooperation

  2025cites this paper
• Culinary Vision-Aided Smart Robotic Arm for Automated Cutting of Kitchen Ingredients

  2025cites this paper
• iOSPointMapper: RealTime Pedestrian and Accessibility Mapping with Mobile AI

  2025cites this paper
• Robotic Motion Intelligence Using Vector Symbolic Architectures and Blockchain-Based Smart Contracts

  2025cites this paper
• A framework for vision-based robotic grasping and sorting in cluttered environments

  2025cites this paper
• The Duo of Visual Servoing and Deep Learning-Based Methods for Situation-Aware Disaster Management: A Comprehensive Review

  2024cites this paper
• Activities of Daily Living Object Dataset: Advancing Assistive Robotic Manipulation with a Tailored Dataset

  2024cites this paper
• Deep Learning for Computer Vision: Innovations in Image Recognition and Processing Techniques

  2024cites this paper
• Aircraft Diagnostics Using Convolutional Neural Networks

  2024cites this paper
• A Review of Machine Learning and Deep Learning for Object Detection, Semantic Segmentation, and Human Action Recognition in Machine and Robotic Vision

  2024cites this paper
• Monocular vision guided deep reinforcement learning UAV systems with representation learning perception

  2023cites this paper
• Hardware Efficient Direct Policy Imitation Learning for Robotic Navigation in Resource-Constrained Settings

  2023cites this paper
• Automated Depth Dataset Generation with Integrated Quality Metrics for Robotic Manipulation

  2023cites this paper
• Data Issues in High-Definition Maps Furniture – A Survey

  2023cites this paper
• Investigation of automation opportunities in warehouse management in construction supply chains using convolutional neural networks

  2023cites this paper
• A Review on the Effectiveness of Machine Learning and Deep Learning Algorithms for Collaborative Robot

  2023cites this paper
• Discretion-Preserving with Data Mining Drive Distribution Scheme with a Universal Social Grid Web for Vans Using Vast Data

  2023cites this paper
• Multi-Site and Multi-Scale Unbalanced Ship Detection Based on CenterNet

  2022cites this paper
• Perceive, Represent, Generate: Translating Multimodal Information to Robotic Motion Trajectories

  2022cites this paper
• Visual Environment perception for obstacle detection and crossing of lower-limb exoskeletons

  2022cites this paper
• The eyes and hearts of UAV pilots: observations of physiological responses in real-life scenarios

  2022cites this paper
• A Survey of the Four Pillars for Small Object Detection: Multiscale Representation, Contextual Information, Super-Resolution, and Region Proposal

  2022cites this paper
• Path Planning of Mobile Robot Based on Improved TD3 Algorithm

  2022cites this paper
• Identification of Incung Characters (Kerinci) to Latin Characters Using Convolutional Neural Network

  2022cites this paper
• Learning A Simulation-based Visual Policy for Real-world Peg In Unseen Holes

  2022cites this paper
• Public Opinion Early Warning Agent Model: A Deep Learning Cascade Virality Prediction Model Based on Multi-Feature Fusion

  2021cites this paper
• Analyzing Adversarial Attacks against Deep Learning for Robot Navigation

  2021cites this paper
• ComBiNet: Compact Convolutional Bayesian Neural Network for Image Segmentation

  2021cites this paper
• Vision System Actualization for Object Localization Robot

  2021cites this paper
• An Overview of Federated Learning at the Edge and Distributed Ledger Technologies for Robotic and Autonomous Systems

  2021cites this paper
• Generic development methodology for flexible robotic pick-and-place workcells based on Digital Twin

  2021cites this paper
• Robotic Grasp Synthesis Using Deep Learning Approaches: A Survey

  2021cites this paper
• The Challenges and Opportunities of Human-Centered AI for Trustworthy Robots and Autonomous Systems

  2021cites this paper
• Robotics Dexterous Grasping: The Methods Based on Point Cloud and Deep Learning

  2021cites this paper
• Application of artificial neural networks and multiple linear regression on local bond stress equation of UHPC and reinforcing steel bars

  2021cites this paper
• Hierarchical deep reinforcement learning to drag heavy objects by adult-sized humanoid robot

  2021cites this paper
• Development of an Autonomous Intelligent Mobile Robot Based on AI and SLAM Technology

  2021cites this paper
• Environment Prediction to Enhance the Navigation System of Water Pipeline Inspection Platforms

  2021cites this paper
• Chapter 5 Robotics as an Enabler of Resiliency to Disasters: Promises and Pitfalls

  2021cites this paper
• Project AI: Real-time object detection and avoidance for autonomous Nao Robots performing in the Standard Platform League

  2021cites this paper
• Fast, yet robust end-to-end camera pose estimation for robotic applications

  2020cites this paper
• Optimized Convolutional Neural Network-Based Object Recognition for Humanoid Robot

  2020cites this paper
• A Formal Account of Structuring Motor Actions With Sensory Prediction for a Naive Agent

  2020cites this paper
• Domain Adaptation from Public Dataset to Robotic Perception Based on Deep Neural Network

  2020cites this paper
• Unsupervised Learning Method for Encoder-Decoder-Based Image Restoration

  2020cites this paper
• Mitigating the catastrophic forgetting effect using generative models

  2020cites this paper
• A Multisensory Learning Architecture for Rotation-invariant Object Recognition

  2020cites this paper
• Object detection networks and augmented reality for cellular detection in fluorescence microscopy

  2020cites this paper
• Survey on artificial intelligence algorithms used in industrial robotics

  2020cites this paper
• Learning to grasp in unstructured environments with deep convolutional neural networks using a Baxter Research Robot

  2019cites this paper
• A real-time human-robot interaction framework with robust background invariant hand gesture detection

  2019cites this paper
• Object Detection Networks and Augmented Reality for Cellular Detection in Fluorescence Microscopy Acquisition and Analysis

  2019cites this paper
• A Review of Deep Learning with Special Emphasis on Architectures, Applications and Recent Trends

  2019cites this paper
• Identifikasi ekspresi wajah berbasis citra menggunakan algoritma Convolutional Neural Network (CNN)

  2019cites this paper
• An affordable deep learning based solution to support pick and place robotic tasks

  2019cites this paper
• Change detection system design using stereo vision for countering improvised explosive devices

  2019cites this paper
• Lessons from reinforcement learning for biological representations of space

  2019cites this paper
• Vision-based Human Gestures Recognition for Human-Robot Interaction. (Reconnaissance des gestes humains basée sur la vision pour l'interaction homme-robot)

  2019cites this paper
• Comprehensive Review of Artificial Neural Network Applications to Pattern Recognition

  2019cites this paper
• Review of Deep Learning Methods in Robotic Grasp Detection

  2018cites this paper
• State-of-the-art in artificial neural network applications: A survey

  2018cites this paper
• A Real-time Physical Human-Robot Interaction Framework with Robust Background Invariant Hand Gesture Detection

  2018cites this paper
• Object detection with artificial vision and neural networks for service robots

  2018cites this paper
• Camera-Based Semantic Enhanced Vehicle Segmentation for Planar LIDAR

  2018cites this paper
• ApproxRobotics: The Cost and Accuracy Tradeoff for Small Mobile AgRobots

  year unknowncites this paper