Learning to perform joint image super-resolution and rain removal via a single-convolutional neural network

Abstract. In outdoor low-level vision systems, not only is the resolution of the imaging system important, but rain corrupts the visibility of outdoor scenes and may cause computer vision systems to fail. We present a deep convolutional neural network (CNN) architecture for simultaneously performing single-image super-resolution and rain removal. Instead of learning an end-to-end mapping between the low-resolution rainy images and high-resolution clean images in the original image space, we train our network in the detail space, i.e., the space obtained by high-pass filtering the original image. The proposed CNN has a lightweight structure, yet it outperforms super-resolution and rain removal consecutively by a significantly large margin (>1  dB on average).

• Publication year

  2018

• Venue

  J. Electronic Imaging

• Publication date

  2018-12-15

• Fields of study

  Computer Science, Engineering, Environmental Science

• Identifiers
  DOI 10.1117/1.JEI.27.6.063024
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Deep joint rain and haze removal from single images

  2018cited by this paper
• Video Super-Resolution via Bidirectional Recurrent Convolutional Networks

  2018influential reference
• Hybrid sparse-representation-based approach to image super-resolution reconstruction

  2017cited by this paper
• Single Image Super-Resolution Using Global Regression Based on Multiple Local Linear Mappings

  2017cited by this paper
• Clearing the Skies: A Deep Network Architecture for Single-Image Rain Removal

  2016cited by this paper
• Deep Joint Rain Detection and Removal from a Single Image

  2016cited by this paper
• Deep Edge Guided Recurrent Residual Learning for Image Super-Resolution

  2016cited by this paper
• Deeply-Recursive Convolutional Network for Image Super-Resolution

  2015cited by this paper
• Deep Networks for Image Super-Resolution with Sparse Prior

  2015cited by this paper
• Utilizing Local Phase Information to Remove Rain from Video

  2015cited by this paper
• Removing Rain from a Single Image via Discriminative Sparse Coding

  2015cited by this paper
• Accurate Image Super-Resolution Using Very Deep Convolutional Networks

  2015cited by this paper
• Learning a Deep Convolutional Network for Image Super-Resolution

  2014cited by this paper
• Face hallucination based on sparse local-pixel structure

  2014influential reference
• Visual Depth Guided Color Image Rain Streaks Removal Using Sparse Coding

  2014cited by this paper
• Self-Learning Based Image Decomposition With Applications to Single Image Denoising

  2014cited by this paper
• Image Super-Resolution Using Deep Convolutional Networks

  2014cited by this paper
• Computer Vision – ECCV 2014

  2014cited by this paper
• Computer Vision – ECCV 2014

  2014cited by this paper
• Single-Image Super-Resolution via Linear Mapping of Interpolated Self-Examples

  2014cited by this paper
• Image Super-Resolution with Fast Approximate Convolutional Sparse Coding

  2014cited by this paper
• A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution

  2014cited by this paper
• Rolling Guidance Filter

  2014cited by this paper
• Single-image deraining using an adaptive nonlocal means filter

  2013cited by this paper
• Nonparametric Blind Super-resolution

  2013cited by this paper
• Restoring an Image Taken through a Window Covered with Dirt or Rain

  2013cited by this paper
• A Generalized Low-Rank Appearance Model for Spatio-temporally Correlated Rain Streaks

  2013cited by this paper
• Anchored Neighborhood Regression for Fast Example-Based Super-Resolution

  2013cited by this paper
• Fast Image Super-Resolution Based on In-Place Example Regression

  2013cited by this paper
• ImageNet classification with deep convolutional neural networks

  2012cited by this paper
• Neural Information Processing

  2012cited by this paper
• Morphable model space based face super-resolution reconstruction and recognition

  2012cited by this paper
• Low-Complexity Single-Image Super-Resolution based on Nonnegative Neighbor Embedding

  2012cited by this paper
• Automatic Single-Image-Based Rain Streaks Removal via Image Decomposition

  2012cited by this paper
• Rain or Snow Detection in Image Sequences Through Use of a Histogram of Orientation of Streaks

  2011cited by this paper
• Contour Detection and Hierarchical Image Segmentation

  2011cited by this paper
• Image Super-Resolution Via Sparse Representation

  2010cited by this paper
• Hallucinating face by position-patch

  2010cited by this paper
• Rectified Linear Units Improve Restricted Boltzmann Machines

  2010cited by this paper
• Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization

  2010cited by this paper
• Guided Image Filtering

  2010cited by this paper
• Super-resolution from a single image

  2009cited by this paper
• Brain Hallucination

  2008cited by this paper
• Computer Vision – ECCV 2008

  2008cited by this paper
• Analysis of Rain and Snow in Frequency Space

  2008cited by this paper
• Hallucinating face by eigentransformation

  2005influential reference
• UCID: an uncompressed color image database

  2003cited by this paper

• Enhanced Attentive Generative Adversarial Network for Single-Image Deraining

  2021cites this paper
• Recursive modified dense network for single-image deraining

  2020influential citation
• Recurrent Attention Dense Network for Single Image De-Raining

  2020cites this paper