InfoMask: Masked Variational Latent Representation to Localize Chest Disease

The scarcity of richly annotated medical images is limiting supervised deep learning based solutions to medical image analysis tasks, such as localizing discriminatory radiomic disease signatures. Therefore, it is desirable to leverage unsupervised and weakly supervised models. Most recent weakly supervised localization methods apply attention maps or region proposals in a multiple instance learning formulation. While attention maps can be noisy, leading to erroneously highlighted regions, it is not simple to decide on an optimal window/bag size for multiple instance learning approaches. In this paper, we propose a learned spatial masking mechanism to filter out irrelevant background signals from attention maps. The proposed method minimizes mutual information between a masked variational representation and the input while maximizing the information between the masked representation and class labels. This results in more accurate localization of discriminatory regions. We tested the proposed model on the ChestX-ray8 dataset to localize pneumonia from chest X-ray images without using any pixel-level or bounding-box annotations.

• Publication year

  2019

• Venue

  International Conference on Medical Image Computing and Computer-Assisted Intervention

• Publication date

  2019-03-28

• Fields of study

  Medicine, Computer Science

• Identifiers
  DOI 10.1007/978-3-030-32226-7_82arXiv 1903.11741
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Weakly Supervised Deep Learning for Whole Slide Lung Cancer Image Analysis

  2020cited by this paper
• Gradient-Based Attribution Methods

  2019cited by this paper
• ChestX-ray: Hospital-Scale Chest X-ray Database and Benchmarks on Weakly Supervised Classification and Localization of Common Thorax Diseases

  2019cited by this paper
• TS2C: Tight Box Mining with Surrounding Segmentation Context for Weakly Supervised Object Detection

  2018cited by this paper
• Weakly Supervised Learning for Whole Slide Lung Cancer Image Classification

  2018cited by this paper
• Weakly Supervised Deep Learning for Thoracic Disease Classification and Localization on Chest X-rays

  2018cited by this paper
• Classifier-agnostic saliency map extraction

  2018cited by this paper
• Adversarial Localization Network

  2017cited by this paper
• Learning Important Features Through Propagating Activation Differences

  2017cited by this paper
• Towards better understanding of gradient-based attribution methods for Deep Neural Networks

  2017cited by this paper
• Deep Variational Information Bottleneck

  2017cited by this paper
• SmoothGrad: removing noise by adding noise

  2017cited by this paper
• CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning

  2017influential reference
• Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization

  2016cited by this paper
• Attention Networks for Weakly Supervised Object Localization

  2016cited by this paper
• Grad-CAM: Why did you say that? Visual Explanations from Deep Networks via Gradient-based Localization

  2016influential reference
• Weakly Supervised Localization Using Deep Feature Maps

  2016cited by this paper
• Weakly Supervised Deep Detection Networks

  2015cited by this paper
• On learning to localize objects with minimal supervision

  2014cited by this paper
• Weakly-supervised Discovery of Visual Pattern Configurations

  2014cited by this paper
• Weakly Supervised Object Localization with Latent Category Learning

  2014cited by this paper
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps

  2013cited by this paper
• Self-Paced Learning for Latent Variable Models

  2010cited by this paper
• Standardization of interpretation of chest radiographs for the diagnosis of pneumonia in children

  2001cited by this paper
• The information bottleneck method

  2000cited by this paper

• Deep Learning Approaches for Medical Imaging Under Varying Degrees of Label Availability: A Comprehensive Survey

  2025cites this paper
• CSSNet: A 3D medical image segmentation network based on compressed sparse dual-branch structure

  2025cites this paper
• From Pixels to Perception: Interpretable Predictions via Instance-wise Grouped Feature Selection

  2025influential citation
• Spatial attention for human-centric visual understanding: An Information Bottleneck method

  2024cites this paper
• From 3D point‐cloud data to explainable geometric deep learning: State‐of‐the‐art and future challenges

  2024cites this paper
• An Improved COVID-19 Classification Model on Chest Radiography by Dual-Ended Multiple Attention Learning

  2023cites this paper
• Recent Advances in Explainable Artificial Intelligence for Magnetic Resonance Imaging

  2023cites this paper
• Disease Localization and Severity Assessment in Chest X-Ray Images using Multi-Stage Superpixels Classification

  2022cites this paper
• Guidelines and evaluation for clinical explainable AI on medical image analysis

  2022cites this paper
• PILLET-GAN: Pixel-Level Lesion Traversal Generative Adversarial Network for Pneumonia Localization

  2022cites this paper
• Guidelines and evaluation of clinical explainable AI in medical image analysis

  2022cites this paper
• Evaluating Explainable AI on a Multi-Modal Medical Imaging Task: Can Existing Algorithms Fulfill Clinical Requirements?

  2022cites this paper
• New Definitions and Evaluations for Saliency Methods: Staying Intrinsic, Complete and Sound

  2022cites this paper
• Active deep learning from a noisy teacher for semi-supervised 3D image segmentation: Application to COVID-19 pneumonia infection in CT

  2022cites this paper
• Consistent response for automated multilabel thoracic disease classification

  2022cites this paper
• One Map Does Not Fit All: Evaluating Saliency Map Explanation on Multi-Modal Medical Images

  2021cites this paper
• Weakly Supervised Thoracic Disease Localization via Disease Masks

  2021cites this paper
• Discriminative Feature Learning for Thorax Disease Classification in Chest X-ray Images

  2021influential citation
• Deep Learning for Chest X-ray Analysis: A Survey

  2021cites this paper
• XProtoNet: Diagnosis in Chest Radiography with Global and Local Explanations

  2021cites this paper
• Latent representation learning in biology and translational medicine

  2021cites this paper
• Explaining COVID-19 and Thoracic Pathology Model Predictions by Identifying Informative Input Features

  2021cites this paper
• Towards Semantic Interpretation of Thoracic Disease and COVID-19 Diagnosis Models

  2021cites this paper
• Robust Representation Learning via Perceptual Similarity Metrics

  2021cites this paper
• Improving Interpretability of Deep Neural Networks in Medical Diagnosis by Investigating the Individual Units

  2021cites this paper
• Information Bottleneck Approach to Spatial Attention Learning

  2021influential citation
• Weakly supervised pneumonia localization in chest x-rays using generative adversarial networks.

  2021cites this paper
• A Coarse Feature Reuse Deep Neural Network for CXR Lesion Detection

  2021cites this paper
• How Do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Masking

  2020cites this paper
• Restricting the Flow: Information Bottlenecks for Attribution

  2020cites this paper
• Flow-Mixup: Classifying Multi-labeled Medical Images with Corrupted Labels

  2020cites this paper
• Explaining by Removing: A Unified Framework for Model Explanation

  2020cites this paper
• Feature Removal Is a Unifying Principle for Model Explanation Methods

  2020cites this paper
• Triple attention learning for classification of 14 thoracic diseases using chest radiography

  2020cites this paper
• SPECHT: Self-tuning Plausibility based object detection Enables quantification of Conflict in Heterogeneous multi-scale microscopy

  2020cites this paper
• Improving Multi-Agent Generative Adversarial Nets with Variational Latent Representation

  2020cites this paper
• Interpreting Graph Neural Networks for NLP With Differentiable Edge Masking

  2020cites this paper
• PointMask: Towards Interpretable and Bias-Resilient Point Cloud Processing

  2020influential citation
• COVID-19 Image Data Collection: Prospective Predictions Are the Future

  2020cites this paper
• Quantification and Analysis of Layer-wise and Pixel-wise Information Discarding

  2019cites this paper
• Information-Bottleneck Approach to Salient Region Discovery

  2019cites this paper
• Deep semantic segmentation of natural and medical images: a review

  2019cites this paper
• Computer Methods and Programs in Biomedicine

  year unknowncites this paper