Dynamic Scale Inference by Entropy Minimization

Given the variety of the visual world there is not one true scale for recognition: objects may appear at drastically different sizes across the visual field. Rather than enumerate variations across filter channels or pyramid levels, dynamic models locally predict scale and adapt receptive fields accordingly. The degree of variation and diversity of inputs makes this a difficult task. Existing methods either learn a feedforward predictor, which is not itself totally immune to the scale variation it is meant to counter, or select scales by a fixed algorithm, which cannot learn from the given task and data. We extend dynamic scale inference from feedforward prediction to iterative optimization for further adaptivity. We propose a novel entropy minimization objective for inference and optimize over task and structure parameters to tune the model to each input. Optimization during inference improves semantic segmentation accuracy and generalizes better to extreme scale variations that cause feedforward dynamic inference to falter.

• Publication year

  2019

• Venue

  arXiv.org

• Publication date

  2019-08-08

• Fields of study

  Computer Science

• Identifiers
  arXiv 1908.03182
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Statistical Learning Theory

  2021cited by this paper
• Semi-Supervised Domain Adaptation via Minimax Entropy

  2019cited by this paper
• Blurring the Line Between Structure and Learning to Optimize and Adapt Receptive Fields

  2019influential reference
• Understanding the impact of entropy on policy optimization

  2018cited by this paper
• ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation

  2018cited by this paper
• Deep Layer Aggregation

  2017cited by this paper
• Deformable Convolutional Networks

  2017influential reference
• On Calibration of Modern Neural Networks

  2017cited by this paper
• FiLM: Visual Reasoning with a General Conditioning Layer

  2017cited by this paper
• Scale-Adaptive Convolutions for Scene Parsing

  2017cited by this paper
• End-to-End Learning for Structured Prediction Energy Networks

  2017cited by this paper
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

  2017cited by this paper
• Automatic differentiation in PyTorch

  2017cited by this paper
• Deep Value Networks Learn to Evaluate and Iteratively Refine Structured Outputs

  2017cited by this paper
• Dynamic Filter Networks

  2016cited by this paper
• Unsupervised Domain Adaptation with Residual Transfer Networks

  2016cited by this paper
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

  2016cited by this paper
• Steerable CNNs

  2016cited by this paper
• Feature Pyramid Networks for Object Detection

  2016cited by this paper
• Human Pose Estimation with Iterative Error Feedback

  2015cited by this paper
• Simultaneous Deep Transfer Across Domains and Tasks

  2015cited by this paper
• Spatial Transformer Networks

  2015cited by this paper
• Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks

  2015cited by this paper
• Recurrent Convolutional Neural Networks for Scene Labeling

  2014cited by this paper
• Locally Scale-Invariant Convolutional Neural Networks

  2014cited by this paper
• Fully convolutional networks for semantic segmentation

  2014cited by this paper
• Adam: A Method for Stochastic Optimization

  2014influential reference
• Some Improvements on Deep Convolutional Neural Network Based Image Classification

  2013cited by this paper
• Visualizing and Understanding Convolutional Networks

  2013cited by this paper
• ImageNet classification with deep convolutional neural networks

  2012cited by this paper
• Distinctive Image Features from Scale-Invariant Keypoints Abstract by Matthijs Dorst Based on the paper by

  2011cited by this paper
• Semantic contours from inverse detectors

  2011cited by this paper
• The Pascal Visual Object Classes (VOC) Challenge

  2010influential reference
• Probabilistic Graphical Models: Principles and Techniques - Adaptive Computation and Machine Learning

  2009cited by this paper
• Graphical Models, Exponential Families, and Variational Inference

  2008cited by this paper
• A Mathematical Theory of Communication

  2006cited by this paper
• A Tutorial on Energy-Based Learning

  2006cited by this paper
• Semi-supervised Learning by Entropy Minimization

  2004cited by this paper
• Distinctive Image Features from Scale-Invariant Keypoints

  2004cited by this paper
• Transductive Inference for Text Classification using Support Vector Machines

  1999cited by this paper
• Gradient-based learning applied to document recognition

  1998cited by this paper
• A neurobiological model of visual attention and invariant pattern recognition based on dynamic routing of information

  1993cited by this paper
• Scale-Space Theory in Computer Vision

  1993cited by this paper
• Function Optimization using Connectionist Reinforcement Learning Algorithms

  1991cited by this paper
• The Laplacian Pyramid as a Compact Image Code

  1983cited by this paper

• A Comprehensive Survey on Test-Time Adaptation Under Distribution Shifts

  2023cites this paper
• Resolution Learning in Deep Convolutional Networks Using Scale-Space Theory

  2021cites this paper
• Tailoring: encoding inductive biases by optimizing unsupervised objectives at prediction time

  2020cites this paper
• Deformable Kernels: Adapting Effective Receptive Fields for Object Deformation

  2019cites this paper
• Under review as a conference paper at ICLR 2020 completely free-form sampling in image space : Spatial Transformers

  2019cites this paper
• Point Cloud Instance Segmentation using Probabilistic Embeddings

  2019cites this paper
• Under review as a conference paper at ICLR 2020 completely free-form sampling in image space : Spatial Transformers

  2019cites this paper