Learning Transferrable Knowledge for Semantic Segmentation with Deep Convolutional Neural Network

We propose a novel weakly-supervised semantic segmentation algorithm based on Deep Convolutional Neural Network (DCNN). Contrary to existing weakly-supervised approaches, our algorithm exploits auxiliary segmentation annotations available for different categories to guide segmentations on images with only image-level class labels. To make segmentation knowledge transferrable across categories, we design a decoupled encoder-decoder architecture with attention model. In this architecture, the model generates spatial highlights of each category presented in images using an attention model, and subsequently performs binary segmentation for each highlighted region using decoder. Combining attention model, the decoder trained with segmentation annotations in different categories boosts accuracy of weakly-supervised semantic segmentation. The proposed algorithm demonstrates substantially improved performance compared to the state-of-theart weakly-supervised techniques in PASCAL VOC 2012 dataset when our model is trained with the annotations in 60 exclusive categories in Microsoft COCO dataset.

• Publication year

  2015

• Venue

  Computer Vision and Pattern Recognition

• Publication date

  2015-12-24

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1109/CVPR.2016.349arXiv 1512.07928
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Decoupled Deep Neural Network for Semi-supervised Semantic Segmentation

  2015influential reference
• Conditional Random Fields as Recurrent Neural Networks

  2015cited by this paper
• BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation

  2015cited by this paper
• Constrained Convolutional Neural Networks for Weakly Supervised Segmentation

  2015cited by this paper
• Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation

  2015influential reference
• Learning Deconvolution Network for Semantic Segmentation

  2015cited by this paper
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention

  2015cited by this paper
• Learning Categories From Few Examples With Multi Model Knowledge Transfer

  2014cited by this paper
• Feedforward semantic segmentation with zoom-out features

  2014cited by this paper
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs

  2014cited by this paper
• Simultaneous Detection and Segmentation

  2014cited by this paper
• Fully convolutional networks for semantic segmentation

  2014influential reference
• Caffe: Convolutional Architecture for Fast Feature Embedding

  2014cited by this paper
• LSDA: Large Scale Detection through Adaptation

  2014cited by this paper
• Recurrent Models of Visual Attention

  2014cited by this paper
• Hypercolumns for object segmentation and fine-grained localization

  2014cited by this paper
• Adam: A Method for Stochastic Optimization

  2014cited by this paper
• Fully Convolutional Multi-Class Multiple Instance Learning

  2014cited by this paper
• Unsupervised Domain Adaptation by Backpropagation

  2014cited by this paper
• Learning to Learn, from Transfer Learning to Domain Adaptation: A Unifying Perspective

  2014cited by this paper
• Microsoft COCO: Common Objects in Context

  2014cited by this paper
• Very Deep Convolutional Networks for Large-Scale Image Recognition

  2014cited by this paper
• From image-level to pixel-level labeling with Convolutional Networks

  2014cited by this paper
• Learning Generative Models with Visual Attention

  2013cited by this paper
• Undoing the Damage of Dataset Bias

  2012cited by this paper
• Searching for objects driven by context

  2012cited by this paper
• Learning to Relate Images

  2011cited by this paper
• Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials

  2011influential reference
• Domain adaptation for object recognition: An unsupervised approach

  2011cited by this paper
• Transfer Learning by Borrowing Examples for Multiclass Object Detection

  2011cited by this paper
• Learning attentional policies for tracking and recognition in video with deep networks

  2011cited by this paper
• Adapting Visual Category Models to New Domains

  2010cited by this paper
• The Pascal Visual Object Classes (VOC) Challenge

  2010cited by this paper
• Learning to combine foveal glimpses with a third-order Boltzmann machine

  2010cited by this paper
• ImageNet: A large-scale hierarchical image database

  2009influential reference
• LabelMe: A Database and Web-Based Tool for Image Annotation

  2008cited by this paper
• A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics

  2001cited by this paper

• Remote-sensing extraction and carbon emission reduction benefit assessment for centralized photovoltaic power plants in Agrivoltaic systems

  2024cites this paper
• Advancing 3D point cloud understanding through deep transfer learning: A comprehensive survey

  2024cites this paper
• Combined Optic Disc and Optic Cup Segmentation Network Based on Adversarial Learning

  2024cites this paper
• Identifying Transfer Learning in the Reshaping of Inductive Biases

  2024cites this paper
• Multi-Source Remote Sensing Images Semantic Segmentation Based on Differential Feature Attention Fusion

  2024cites this paper
• CyCSNet: Learning Cycle-Consistency of Semantics for Weakly-Supervised Semantic Segmentation

  2024influential citation
• DHR: Dual Features-Driven Hierarchical Rebalancing in Inter- and Intra-Class Regions for Weakly-Supervised Semantic Segmentation

  2024cites this paper
• Adjustable patch and feature prior token-based transformer for weakly supervised semantic segmentation

  2024cites this paper
• A Multilabel Text Classifier of Cancer Literature at the Publication Level: Methods Study of Medical Text Classification

  2023cites this paper
• Review on chest pathogies detection systems using deep learning techniques

  2023cites this paper
• Convolutional Neural Networks with Modified Evaluation Functions for Image Semantic Segmentation

  2023cites this paper
• Face Positioned Driver Drowsiness Detection Using Multistage Adaptive 3D Convolutional Neural Network

  2023influential citation
• High-Resolution Network with Dynamic Convolution and Coordinate Attention for Classification of Chest X-ray Images

  2023cites this paper
• Multi-Granularity Denoising and Bidirectional Alignment for Weakly Supervised Semantic Segmentation

  2023cites this paper
• An Improved COVID-19 Classification Model on Chest Radiography by Dual-Ended Multiple Attention Learning

  2023cites this paper
• MARS: Model-agnostic Biased Object Removal without Additional Supervision for Weakly-Supervised Semantic Segmentation

  2023cites this paper
• Semantic Image Segmentation: Two Decades of Research

  2023cites this paper
• PanopticNeRF-360: Panoramic 3D-to-2D Label Transfer in Urban Scenes

  2023cites this paper
• Segmentation Based on 1-Shot Learning

  2023cites this paper
• Real-Time Aerial Detection and Reasoning on Embedded-UAVs in Rural Environments

  2023cites this paper
• Coupling Global Context and Local Contents for Weakly-Supervised Semantic Segmentation

  2023cites this paper
• Mining semantic information from intra-image and cross-image for few-shot segmentation

  2022cites this paper
• A Deep Reinforcement Learning Approach for Crowdshipping Vehicle Routing Problem

  2022cites this paper
• Incorporating Self Attention Mechanism into Polarization spectral image Reflection Network

  2022cites this paper
• Deep Bilateral Filtering Network for Point-Supervised Semantic Segmentation in Remote Sensing Images

  2022cites this paper
• Semi-Supervised Instance-Segmentation Model for Feature Transfer Based on Category Attention

  2022cites this paper
• Charge Own Job: Saliency Map and Visual Word Encoder for Image-Level Semantic Segmentation

  2022cites this paper
• Contour-enhanced attention CNN for CT-based COVID-19 segmentation

  2022cites this paper
• Semisupervised representative learning for measuring epidermal thickness in human subjects in optical coherence tomography by leveraging datasets from rodent models

  2022cites this paper
• RecurSeed and EdgePredictMix: Pseudo-Label Refinement Learning for Weakly Supervised Semantic Segmentation across Single- and Multi-Stage Frameworks

  2022cites this paper
• Deep Active Contour-Based Capsule Network for Medical Image Segmentation

  2022cites this paper
• Bag-level classification network for infrared target detection

  2022cites this paper
• Research Convey on Text Classification Method based on Deep Learning

  2022cites this paper
• RecurSeed and CertainMix for Weakly Supervised Semantic Segmentation

  2022cites this paper
• Panoptic NeRF: 3D-to-2D Label Transfer for Panoptic Urban Scene Segmentation

  2022cites this paper
• Representation Compensation Networks for Continual Semantic Segmentation

  2022cites this paper
• Transfer Learning Based Semantic Segmentation for 3D Object Detection from Point Cloud

  2021cites this paper
• Identifying fraud in medical insurance based on blockchain and deep learning

  2021cites this paper
• GETAM: Gradient-weighted Element-wise Transformer Attention Map for Weakly-supervised Semantic segmentation

  2021cites this paper
• ECSD-Net: A joint optic disc and cup segmentation and glaucoma classification network based on unsupervised domain adaptation

  2021cites this paper
• Learning or Modelling? An Analysis of Single Image Segmentation Based on Scribble Information

  2021cites this paper
• DGFNet: Dual Gate Fusion Network for Land Cover Classification in Very High-Resolution Images

  2021cites this paper
• Spatial Resolution Enhancement of Brillouin Optical Correlation-Domain Reflectometry Using Convolutional Neural Network: Proof of Concept

  2021cites this paper
• Adaptive Affinity Loss and Erroneous Pseudo-Label Refinement for Weakly Supervised Semantic Segmentation

  2021cites this paper
• SACNN: Self-attentive Convolutional Neural Network Model for Natural Language Inference

  2021cites this paper
• An Overview and Experimental Study of Learning-Based Optimization Algorithms for the Vehicle Routing Problem

  2021cites this paper
• Towards Single Stage Weakly Supervised Semantic Segmentation

  2021cites this paper
• Learning from unlabelled real seismic data: Fault detection based on transfer learning

  2021cites this paper
• Learning Class-Agnostic Pseudo Mask Generation for Box-Supervised Semantic Segmentation

  2021influential citation
• Weakly-Supervised Cross-Domain Road Scene Segmentation via Multi-Level Curriculum Adaptation

  2021cites this paper
• Weakly-supervised Object Representation Learning for Few-shot Semantic Segmentation

  2021cites this paper
• Atrous convolutional feature network for weakly supervised semantic segmentation

  2021cites this paper
• Driver Drowsiness Estimation Based on Factorized Bilinear Feature Fusion and a Long-Short-Term Recurrent Convolutional Network

  2020cites this paper
• On the Effective Transfer Learning Strategy for Medical Image Analysis in Deep Learning

  2020cites this paper
• Deep Transferring Quantization

  2020cites this paper
• Single-Stage Semantic Segmentation From Image Labels

  2020cites this paper
• Dense Attention Convolutional Network for Image Classification

  2020cites this paper
• Thorax-Net: An Attention Regularized Deep Neural Network for Classification of Thoracic Diseases on Chest Radiography

  2020cites this paper
• Learning to Discover Multi-Class Attentional Regions for Multi-Label Image Recognition

  2020cites this paper
• Intégration de Connaissances aux Modèles Neuronaux pour la Détection de Relations Visuelles Rares. (Knowledge Integration into Neural Networks for the purposes of Rare Visual Relation Detection)

  2020cites this paper
• Automatic monitoring of steel strip positioning error based on semantic segmentation

  2020cites this paper
• Scale-Aware Feature Network for Weakly Supervised Semantic Segmentation

  2020cites this paper
• Are We Hungry for 3D LiDAR Data for Semantic Segmentation?

  2020cites this paper
• Synthesizing Supervision for Learning Deep Saliency Network without Human Annotation

  2020cites this paper
• Simplified unsupervised image translation for semantic segmentation adaptation

  2020cites this paper
• Triple attention learning for classification of 14 thoracic diseases using chest radiography

  2020cites this paper
• Are We Hungry for 3D LiDAR Data for Semantic Segmentation? A Survey and Experimental Study

  2020cites this paper
• Strip Pooling: Rethinking Spatial Pooling for Scene Parsing

  2020cites this paper
• Dual-Level Selective Transfer Learning for Intrahepatic Cholangiocarcinoma Segmentation in Non-enhanced Abdominal CT

  2020cites this paper
• Context propagation embedding network for weakly supervised semantic segmentation

  2020cites this paper
• Weakly Supervised Semantic Segmentation with Boundary Exploration

  2020cites this paper
• Weakly Supervised Semantic Segmentation by a Class-Level Multiple Group Cosegmentation and Foreground Fusion Strategy

  2020cites this paper
• Dynamic attention network for semantic segmentation

  2020cites this paper
• VOSTR: Video Object Segmentation via Transferable Representations

  2020influential citation
• Semantic Segmentation Based on Convolution Neural Network for Steel Strip Position Estimation

  2020cites this paper
• Multi-Source Deep Domain Adaptation with Weak Supervision for Time-Series Sensor Data

  2020cites this paper
• Guided Attention Inference Network

  2020cites this paper
• Multi-Organ Segmentation Over Partially Labeled Datasets With Multi-Scale Feature Abstraction

  2020cites this paper
• Multi-Label Image Recognition with Multi-Class Attentional Regions

  2020cites this paper
• FaHo: deep learning enhanced holographic localization for RFID tags

  2019cites this paper
• Deep Max-Margin Discriminant Projection

  2019cites this paper
• Single Image Reflection Removal Using Convolutional Neural Networks

  2019cites this paper
• Progressive Generative Adversarial Networks for Medical Image Super resolution

  2019cites this paper
• FickleNet: Weakly and Semi-Supervised Semantic Image Segmentation Using Stochastic Inference

  2019influential citation
• Driver Drowsiness Detection Using Condition-Adaptive Representation Learning Framework

  2019cites this paper
• Large-Scale Weakly-Supervised Pre-Training for Video Action Recognition

  2019cites this paper
• Hierarchical deep transfer learning for fine-grained categorization on micro datasets

  2019cites this paper
• Multi-Round Transfer Learning for Low-Resource NMT Using Multiple High-Resource Languages

  2019cites this paper
• Semantic Segmentation with Transfer Learning for Off-Road Autonomous Driving

  2019cites this paper
• Analyzing the Cross-Sensor Portability of Neural Network Architectures for LiDAR-based Semantic Labeling

  2019cites this paper
• Multifaceted Analysis of Fine-Tuning in a Deep Model for Visual Recognition

  2019cites this paper
• Frame-to-Frame Aggregation of Active Regions in Web Videos for Weakly Supervised Semantic Segmentation

  2019cites this paper
• Customizing Student Networks From Heterogeneous Teachers via Adaptive Knowledge Amalgamation

  2019cites this paper
• Fast Video Object Segmentation via Mask Transfer Network

  2019cites this paper
• Spatial Structure Preserving Feature Pyramid Network for Semantic Image Segmentation

  2019cites this paper
• AdaRF

  2019cites this paper
• Action matching network: open-set action recognition using spatio-temporal representation matching

  2019cites this paper
• Training Deep Neural Networks with Adversarially Augmented Features for Small-scale Training Datasets

  2019cites this paper
• Deep Semantic Parsing of Freehand Sketches With Homogeneous Transformation, Soft-Weighted Loss, and Staged Learning

  2019cites this paper
• Cross-Modal Attention-Guided Convolutional Network for Multi-modal Cardiac Segmentation

  2019cites this paper