Adapting Deep Visuomotor Representations with Weak Pairwise Constraints

Real-world robotics problems often occur in domains that differ significantly from the robot's prior training environment. For many robotic control tasks, real world experience is expensive to obtain, but data is easy to collect in either an instrumented environment or in simulation. We propose a novel domain adaptation approach for robot perception that adapts visual representations learned on a large easy-to-obtain source dataset (e.g. synthetic images) to a target real-world domain, without requiring expensive manual data annotation of real world data before policy search. Supervised domain adaptation methods minimize cross-domain differences using pairs of aligned images that contain the same object or scene in both the source and target domains, thus learning a domain-invariant representation. However, they require manual alignment of such image pairs. Fully unsupervised adaptation methods rely on minimizing the discrepancy between the feature distributions across domains. We propose a novel, more powerful combination of both distribution and pairwise image alignment, and remove the requirement for expensive annotation by using weakly aligned pairs of images in the source and target domains. Focusing on adapting from simulation to real world data using a PR2 robot, we evaluate our approach on a manipulation task and show that by using weakly paired images, our method compensates for domain shift more effectively than previous techniques, enabling better robot performance in the real world.

• Publication year

  2015

• Venue

  Workshop on the Algorithmic Foundations of Robotics

• Publication date

  2015-11-23

• Fields of study

  Computer Science, Engineering

• Identifiers
  DOI 10.1007/978-3-030-43089-4_44arXiv 1511.07111
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Learning Transferable Policies for Monocular Reactive MAV Control

  2016cited by this paper
• Simultaneous Deep Transfer Across Domains and Tasks

  2015influential reference
• Deep spatial autoencoders for visuomotor learning

  2015influential reference
• Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images

  2015cited by this paper
• ALVINN, an autonomous land vehicle in a neural network

  2015cited by this paper
• End-to-End Training of Deep Visuomotor Policies

  2015influential reference
• Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours

  2015cited by this paper
• Continuous control with deep reinforcement learning

  2015cited by this paper
• Trust Region Policy Optimization

  2015cited by this paper
• Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views

  2015cited by this paper
• Towards Vision-Based Deep Reinforcement Learning for Robotic Motion Control

  2015cited by this paper
• Learning Transferable Features with Deep Adaptation Networks

  2015cited by this paper
• Towards Adapting Deep Visuomotor Representations from Simulated to Real Environments

  2015influential reference
• Learning Deep Object Detectors from 3D Models

  2014cited by this paper
• Exploring Invariances in Deep Convolutional Neural Networks Using Synthetic Images

  2014cited by this paper
• Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation

  2014cited by this paper
• ImageNet Large Scale Visual Recognition Challenge

  2014cited by this paper
• From Virtual to Reality: Fast Adaptation of Virtual Object Detectors to Real Domains

  2014cited by this paper
• Unsupervised Domain Adaptation by Backpropagation

  2014cited by this paper
• Beyond PASCAL: A benchmark for 3D object detection in the wild

  2014influential reference
• Caffe: Convolutional Architecture for Fast Feature Embedding

  2014cited by this paper
• Deep Domain Confusion: Maximizing for Domain Invariance

  2014cited by this paper
• DeepPose: Human Pose Estimation via Deep Neural Networks

  2013cited by this paper
• Learning task error models for manipulation

  2013cited by this paper
• Playing Atari with Deep Reinforcement Learning

  2013cited by this paper
• Efficient Learning of Domain-invariant Image Representations

  2013cited by this paper
• Geodesic flow kernel for unsupervised domain adaptation

  2012cited by this paper
• Learning with Augmented Features for Heterogeneous Domain Adaptation

  2012cited by this paper
• Autonomous reinforcement learning on raw visual input data in a real world application

  2012cited by this paper
• Domain adaptation for object recognition: An unsupervised approach

  2011cited by this paper
• What you saw is not what you get: Domain adaptation using asymmetric kernel transforms

  2011cited by this paper
• Tabula rasa: Model transfer for object category detection

  2011cited by this paper
• The Pascal Visual Object Classes (VOC) Challenge

  2010cited by this paper
• Pose-Sensitive Embedding by Nonlinear NCA Regression

  2010cited by this paper
• Adapting Visual Category Models to New Domains

  2010cited by this paper
• Learning and Domain Adaptation

  2009cited by this paper
• 3D laser scan classification using web data and domain adaptation

  2009cited by this paper
• Learning and generalization of motor skills by learning from demonstration

  2009cited by this paper
• Domain Adaptation: Learning Bounds and Algorithms

  2009cited by this paper
• Sparse probabilistic regression for activity-independent human pose inference

  2008cited by this paper
• Robotic Grasping of Novel Objects using Vision

  2008cited by this paper
• Cross-domain video concept detection using adaptive svms

  2007cited by this paper
• Frustratingly Easy Domain Adaptation

  2007cited by this paper
• Dimensionality Reduction by Learning an Invariant Mapping

  2006cited by this paper
• Learning a similarity metric discriminatively, with application to face verification

  2005cited by this paper
• Fast pose estimation with parameter-sensitive hashing

  2003cited by this paper
• Signature Verification Using A "Siamese" Time Delay Neural Network

  1993cited by this paper
• The ACRONYM Model-Based Vision System

  1979cited by this paper

• Self-Expert Imitation With Purifying Latent Feature for Generalization in Visual Reinforcement Learning

  2026cites this paper
• AMM: Adaptive Modularized Reinforcement Model for Multi-city Traffic Signal Control

  2025cites this paper
• DiG-Flow: Discrepancy-Guided Flow Matching for Robust VLA Models

  2025cites this paper
• SIMS-V: Simulated Instruction-Tuning for Spatial Video Understanding

  2025cites this paper
• Real-Time Performance Evaluation of Household Object Detection, Recognition and Grasping Using Firebird V Robot

  2025cites this paper
• Understanding and Robustifying Sub-domain Alignment for Domain Adaptation

  2025cites this paper
• cMALC-D: Contextual Multi-Agent LLM-Guided Curriculum Learning with Diversity-Based Context Blending

  2025cites this paper
• Federated Deep Reinforcement Learning for Task Scheduling in Heterogeneous Autonomous Robotic System

  2024cites this paper
• Synthesizing Programmatic Policy for Generalization within Task Domain

  2024cites this paper
• Low-Rank Optimal Transport for Robust Domain Adaptation

  2024cites this paper
• Transferable Hierarchical Skill Learning with Diffusion Models for Robot Manipulation

  2024cites this paper
• Domain Adaptation in Visual Reinforcement Learning via Self-Expert Imitation with Purifying Latent Feature

  2024cites this paper
• An Approach Combining Feature Selection with Machine Learning Techniques for Prediction Reliability and Accuracy in Hepatitis Diagnosis

  2023cites this paper
• Domain Adaptation via Rebalanced Sub-domain Alignment

  2023cites this paper
• Sim-to-real via latent prediction: Transferring visual non-prehensile manipulation policies

  2023cites this paper
• Visual Navigation Subject to Embodied Mismatch

  2023cites this paper
• P3C: A Real-World Scenario Dataset Generating Pipeline for Learning-Based Planning and Control Algorithm

  2023cites this paper
• Machine Learning Meets Advanced Robotic Manipulation

  2023cites this paper
• Sim-to-Real Deep Reinforcement Learning with Manipulators for Pick-and-Place

  2023cites this paper
• Cross-Domain Requirements Linking via Adversarial-based Domain Adaptation

  2023cites this paper
• How Simulation Helps Autonomous Driving: A Survey of Sim2real, Digital Twins, and Parallel Intelligence

  2023cites this paper
• DiAReL: Reinforcement Learning With Disturbance Awareness for Robust Sim2Real Policy Transfer in Robot Control

  2023cites this paper
• Online Prototype Alignment for Few-shot Policy Transfer

  2023influential citation
• Building Manufacturing Deep Learning Models with Minimal and Imbalanced Training Data Using Domain Adaptation and Data Augmentation

  2023cites this paper
• Sim2real and Digital Twins in Autonomous Driving: A Survey

  2023cites this paper
• Progressive Transfer Learning for Dexterous In-Hand Manipulation With Multifingered Anthropomorphic Hand

  2023cites this paper
• HaDR: Applying Domain Randomization for Generating Synthetic Multimodal Dataset for Hand Instance Segmentation in Cluttered Industrial Environments

  2023cites this paper
• Training and Evaluation of Deep Policies using Reinforcement Learning and Generative Models

  2022cites this paper
• CoBeL-RL: A neuroscience-oriented simulation framework for complex behavior and learning

  2022cites this paper
• CACTI: A Framework for Scalable Multi-Task Multi-Scene Visual Imitation Learning

  2022cites this paper
• Multi-Environment Pretraining Enables Transfer to Action Limited Datasets

  2022cites this paper
• Sim2Real image translation to improve a synthetic dataset for a bin picking task

  2022cites this paper
• Few-Shot Image-to-Semantics Translation for Policy Transfer in Reinforcement Learning

  2022cites this paper
• Fine-tuning Deep Reinforcement Learning Policies with r-STDP for Domain Adaptation

  2022cites this paper
• Specific emitter identification based on the multi‐discrepancy deep adaptation network

  2022cites this paper
• A Survey on Adversarial Domain Adaptation

  2022cites this paper
• DayDreamer: World Models for Physical Robot Learning

  2022cites this paper
• Benchmarking domain adaptation for semantic segmentation

  2022cites this paper
• Weakly Supervised Correspondence Learning

  2022cites this paper
• E2HRL: An Energy-efficient Hardware Accelerator for Hierarchical Deep Reinforcement Learning

  2022cites this paper
• Explorative Experiments: A Paradigm Shift to Deal with Severe Uncertainty in Autonomous Robotics

  2022cites this paper
• Review on generic methods for mechanical modeling, simulation and control of soft robots

  2022cites this paper
• PPR-Net++: Accurate 6-D Pose Estimation in Stacked Scenarios

  2022cites this paper
• Learning from Imperfect Demonstrations via Adversarial Confidence Transfer

  2022cites this paper
• Benchmarking Domain Adaptation Methods on Aerial Datasets

  2021cites this paper
• A Multi-Stage Approach Combining Feature Selection with Machine Learning Techniques for Higher Prediction Reliability and Accuracy in Cervical Cancer Diagnosis

  2021cites this paper
• ADA-AT/DT: An Adversarial Approach for Cross-Domain and Cross-Task Knowledge Transfer

  2021cites this paper
• A Survey of Sim-to-Real Transfer Techniques Applied to Reinforcement Learning for Bioinspired Robots

  2021cites this paper
• On Assessing the Usefulness of Proxy Domains for Developing and Evaluating Embodied Agents

  2021cites this paper
• SIM2REALVIZ: Visualizing the Sim2Real Gap in Robot Ego-Pose Estimation

  2021cites this paper
• OHPL: One-shot Hand-eye Policy Learner

  2021cites this paper
• Transfer learning of coverage functions via invariant properties in the fourier domain

  2021cites this paper
• Domain Adaptation In Reinforcement Learning Via Latent Unified State Representation

  2021cites this paper
• ERMAS: Becoming Robust to Reward Function Sim-to-Real Gaps in Multi-Agent Simulations

  2021cites this paper
• Reinforcement learning applied to machine vision: state of the art

  2021cites this paper
• Partially observable environment estimation with uplift inference for reinforcement learning based recommendation

  2021cites this paper
• The Societal Implications of Deep Reinforcement Learning

  2021cites this paper
• An Energy Efficient EdgeAI Autoencoder Accelerator for Reinforcement Learning

  2021cites this paper
• Domain Adaptive Egocentric Person Re-identification

  2021cites this paper
• Learning for a Robot: Deep Reinforcement Learning, Imitation Learning, Transfer Learning

  2021cites this paper
• Learning Deep Features for Shape Correspondence with Domain Invariance

  2021cites this paper
• Controlling by Showing: i-Mimic: A Video-Based Method to Control Robotic Arms

  2021cites this paper
• Off Environment Evaluation Using Convex Risk Minimization

  2021cites this paper
• Unsupervised Domain Adaptation for Visual Navigation

  2020cites this paper
• Domain Adaptation for Visual Understanding

  2020cites this paper
• End to End Learning in Autonomous Driving Systems

  2020cites this paper
• The Ingredients of Real-World Robotic Reinforcement Learning

  2020cites this paper
• Visual Transfer for Reinforcement Learning via Wasserstein Domain Confusion

  2020influential citation
• Provably Efficient Model-based Policy Adaptation

  2020cites this paper
• C2DAN: An Improved Deep Adaptation Network with Domain Confusion and Classifier Adaptation

  2020cites this paper
• Unseen Object Instance Segmentation for Robotic Environments

  2020cites this paper
• Data-efficient visuomotor policy training using reinforcement learning and generative models

  2020cites this paper
• Modular transfer learning with transition mismatch compensation for excessive disturbance rejection

  2020cites this paper
• Robust Sim2Real Transfer by Learning Inverse Dynamics of Simulated Systems

  2020cites this paper
• Deep Adversarial Transition Learning using Cross-Grafted Generative Stacks

  2020cites this paper
• Understanding Unnatural Questions Improves Reasoning over Text

  2020influential citation
• Localising In Complex Scenes Using Balanced Adversarial Adaptation

  2020cites this paper
• Reinforcement Learning with Videos: Combining Offline Observations with Interaction

  2020cites this paper
• Learning Cross-Domain Correspondence for Control with Dynamics Cycle-Consistency

  2020cites this paper
• Policy Distillation from World Models

  2020cites this paper
• Unsupervised Domain Adaptation for Visual Navigation

  2020cites this paper
• Implementation of the perfect plasticity approximation with biogeochemical compartments in R

  2020cites this paper
• Real-World Robotic Perception and Control Using Synthetic Data

  2019cites this paper
• Domain Invariant and Semantic Aware Visual Servoing

  2019cites this paper
• Pontryagin Differentiable Programming: An End-to-End Learning and Control Framework

  2019cites this paper
• Adversarial Feature Training for Generalizable Robotic Visuomotor Control

  2019cites this paper
• Generalization in transfer learning: robust control of robot locomotion

  2019cites this paper
• Unsupervised Domain Adaptation with Shared Latent Dynamics for Reinforcement Learning

  2019cites this paper
• Initial Mask Processor Open / Close Morphological Transform Closest Connected Component RGB Refinement Network Refined Masks Depth Seeding Network Foreground Center Directions Depth Initial Masks RGB

  2019cites this paper
• Few-Shot Meta-Denoising

  2019cites this paper
• The Best of Both Modes: Separately Leveraging RGB and Depth for Unseen Object Instance Segmentation

  2019cites this paper
• End-to-end nonprehensile rearrangement with deep reinforcement learning and simulation-to-reality transfer

  2019cites this paper
• Sim2Real Viewpoint Invariant Visual Servoing by Recurrent Control

  2018cites this paper
• Self-Similarity Grouping: A Simple Unsupervised Cross Domain Adaptation Approach for Person Re-Identification

  2018cites this paper
• Improving Deep Reinforcement Learning Using Graph Convolution and Visual Domain Transfer

  2018cites this paper
• CRAVES: Controlling Robotic Arm With a Vision-Based Economic System

  2018cites this paper
• Implementation of the Perfect Plasticity Approximation with biogeochemical compartments in R

  2018cites this paper
• Visual Data Simulation for Deep Learning in Robot Manipulation Tasks

  2018cites this paper
• Deep learning model-based algorithm for SAR ATR

  2018cites this paper
• Comparing Task Simplifications to Learn Closed-Loop Object Picking Using Deep Reinforcement Learning

  2018cites this paper