Not Just a Black Box: Learning Important Features Through Propagating Activation Differences

Note: This paper describes an older version of DeepLIFT. See this https URL for the newer version. Original abstract follows: The purported "black box" nature of neural networks is a barrier to adoption in applications where interpretability is essential. Here we present DeepLIFT (Learning Important FeaTures), an efficient and effective method for computing importance scores in a neural network. DeepLIFT compares the activation of each neuron to its 'reference activation' and assigns contribution scores according to the difference. We apply DeepLIFT to models trained on natural images and genomic data, and show significant advantages over gradient-based methods.

• Publication year

  2016

• Venue

  arXiv.org

• Publication date

  2016-05-05

• Fields of study

  Computer Science

• Identifiers
  arXiv 1605.01713
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation

  2015cited by this paper
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling

  2014cited by this paper
• Striving for Simplicity: The All Convolutional Net

  2014cited by this paper
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps

  2013cited by this paper
• Visualizing and Understanding Convolutional Networks

  2013cited by this paper
• Deconvolutional networks

  2010cited by this paper
• Long Short-Term Memory

  1997cited by this paper

• Forecasting Equity Correlations with Hybrid Transformer Graph Neural Network

  2026cites this paper
• Quantifying Explanation Quality in Graph Neural Networks using Out-of-Distribution Generalization

  2026cites this paper
• iFlip: Iterative Feedback-driven Counterfactual Example Refinement

  2026cites this paper
• Exploiting Shared Adversarial Features for Dynamic Attacks in Large Vision-Language Models

  2026cites this paper
• Adversarial Evasion Attacks on Computer Vision using SHAP Values

  2026cites this paper
• Causality is Key for Interpretability Claims to Generalise

  2026cites this paper
• Action Shapley: A Training Data Selection Metric for World Model in Reinforcement Learning

  2026cites this paper
• Machine Learning in Epidemiology

  2026cites this paper
• DeepLeak: Privacy Enhancing Hardening of Model Explanations Against Membership Leakage

  2026cites this paper
• Enhancing randomized recurrent neural networks with explainable attribution methods

  2026cites this paper
• On the Necessity of Multi-Domain Explanation: An Uncertainty Principle Approach for Deep Time Series Models

  2025cites this paper
• Explaining BiLSTM-based Fiber Optic Prediction Based on Integrated Gradients

  2025cites this paper
• Fragment-Wise Interpretability in Graph Neural Networks via Molecule Decomposition and Contribution Analysis

  2025influential citation
• A review of explainable artificial intelligence in smart manufacturing

  2025cites this paper
• Evaluating the robustness of explainable AI in medical image recognition under natural and adversarial data corruption

  2025cites this paper
• SSplain: Sparse and Smooth Explainer for Retinopathy of Prematurity Classification

  2025influential citation
• Explainability of Protein Deep Learning Models

  2025cites this paper
• Learning Conformal Explainers for Image Classifiers

  2025cites this paper
• Pulling Back the Curtain on Deep Networks

  2025cites this paper
• Probing Deep into Temporal Profile Makes the Infrared Small Target Detector Much Better

  2025cites this paper
• XAIpipeline: Automated Orchestration of Explainable AI Services for Cloud AI and Open-source Models

  2025cites this paper
• Privacy Preservation with Noise in Explainable AI

  2025cites this paper
• Transforming Data Representation: A Comparative Analysis of Tabular and Image-Based Approaches with XAI

  2025cites this paper
• Informing EEG-Based Error Decoding With Explainable AI

  2025cites this paper
• An Explainable AI Exploration of the Machine Learning Classification of Neoplastic Intracerebral Hemorrhage from Non-Contrast CT

  2025cites this paper
• Explainable AI Methods for Underwater Mine Warfare

  2025cites this paper
• An Explainable Deep-Learning Approach to Detect Pediatric Sleep Apnea From Single-Channel Airflow

  2025cites this paper
• Hierarchical Language Model Design For Interpretable Graph Reasoning

  2025cites this paper
• Explainable Graph Neural Networks for Power Grid Fault Detection

  2025cites this paper
• Understanding the Black Box: A Deep Empirical Dive into Shapley Value Approximations for Tabular Data

  2025cites this paper
• Analysis of Visual Explainers Using Clustering Techniques

  2025cites this paper
• Analysis of slope instability factors: An application study of a novel interpretable ensemble model

  2025cites this paper
• High-resolution reconstruction of cell-type specific transcriptional regulatory processes from bulk sequencing samples

  2025cites this paper
• Visagreement: Visualizing and Exploring Explanations (Dis)Agreement

  2025cites this paper
• Application of explainable machine learning to eXtreme Gradient Boosting models of all-optical logic gates

  2025cites this paper
• GIM: Improved Interpretability for Large Language Models

  2025cites this paper
• Privacy Risks and Preservation Methods in Explainable Artificial Intelligence: A Scoping Review

  2025cites this paper
• EPIC: Explanation of Pretrained Image Classification Networks via Prototype

  2025cites this paper
• XAI-Units: Benchmarking Explainability Methods with Unit Tests

  2025cites this paper
• Evaluating LLMs Robustness in Less Resourced Languages with Proxy Models

  2025cites this paper
• A comprehensive analysis of perturbation methods in explainable AI feature attribution validation for neural time series classifiers

  2025cites this paper
• User Context Generation for Large Language Models From Mobile Sensing Data

  2025cites this paper
• DocVCE: Diffusion-based Visual Counterfactual Explanations for Document Image Classification

  2025cites this paper
• Self-Explaining Reinforcement Learning for Mobile Network Resource Allocation

  2025cites this paper
• Fine-grained Analysis of Brain-LLM Alignment through Input Attribution

  2025influential citation
• Learning from Sufficient Rationales: Analysing the Relationship Between Explanation Faithfulness and Token-level Regularisation Strategies

  2025cites this paper
• Explainable Bayesian deep learning through input-skip Latent Binary Bayesian Neural Networks

  2025cites this paper
• Interpreting CLIP with Hierarchical Sparse Autoencoders

  2025cites this paper
• Securing Virtual Reality Experiences: Unveiling and Tackling Cybersickness Attacks With Explainable AI

  2025cites this paper
• Inducing Implicit Focus Through Structured Irrelevance for Robust Neural PDE Solvers

  2025cites this paper
• Time-series attribution maps with regularized contrastive learning

  2025cites this paper
• Gradient-based Explanations for Deep Learning Survival Models

  2025cites this paper
• Using the Path of Least Resistance to Explain Deep Networks

  2025cites this paper
• Towards Better Interpretability of Sepsis Prediction by Deep Neural Networks with Variable-Wise Attribution Maps

  2025cites this paper
• B-XAIC Dataset: Benchmarking Explainable AI for Graph Neural Networks Using Chemical Data

  2025cites this paper
• Feature-Guided Analysis of Neural Networks: A Replication Study

  2025cites this paper
• Generating and leveraging explanations of AI/ML models in materials and manufacturing research

  2025cites this paper
• TDHook: A Lightweight Framework for Interpretability

  2025cites this paper
• On the Complexity-Faithfulness Trade-off of Gradient-Based Explanations

  2025cites this paper
• Rethinking Robustness: A New Approach to Evaluating Feature Attribution Methods

  2025cites this paper
• Comprehensive Threat Validation for Enhanced Network Security

  2025cites this paper
• VDPST: Enhancing Line-Level Vulnerability Detection with Patch Slicing and Transformers

  2025cites this paper
• On Spectral Properties of Gradient-Based Explanation Methods

  2025cites this paper
• Bioinspired Tactile Object Identification Leveraging Deep Learning and Soft Body Compliance

  2025cites this paper
• From Abstract to Actionable: Pairwise Shapley Values for Explainable AI

  2025cites this paper
• Gradient Co-occurrence Analysis for Detecting Unsafe Prompts in Large Language Models

  2025cites this paper
• Accuracy and Scalability of Machine Learning Methods for Genotype-Phenotype Association Data

  2025cites this paper
• A semi-supervised active learning approach for block-status classification

  2025cites this paper
• Evaluating Interpretable Methods via Geometric Alignment of Functional Distortions

  2025cites this paper
• Tangentially Aligned Integrated Gradients for User-Friendly Explanations

  2025cites this paper
• Honey Bee Piping Detection Utilizing Convolutional Neural Networks

  2025cites this paper
• Heuristic XAI: AIME and HMM Integration Improved-Feature-Importance Assessment Through Heuristics

  2025cites this paper
• Uncertainty Propagation in XAI: A Comparison of Analytical and Empirical Estimators

  2025cites this paper
• Task-Circuit Quantization: Leveraging Knowledge Localization and Interpretability for Compression

  2025influential citation
• OpDiag: Unveiling Database Performance Anomalies Through Query Operator Attribution

  2025cites this paper
• Recovering S-Box Design Structures and Quantifying Distances between S-Boxes using Deep Learning

  2025cites this paper
• Implet: A Post-hoc Subsequence Explainer for Time Series Models

  2025cites this paper
• An Innovative Deep Learning Approach for Ventilator-Associated Pneumonia (VAP) Prediction in Intensive Care Units—Pneumonia Risk Evaluation and Diagnostic Intelligence via Computational Technology (PREDICT)

  2025cites this paper
• Pathwise Explanation of ReLU Neural Networks

  2025cites this paper
• Unlearning-based Neural Interpretations

  2024influential citation
• INFORMER- Interpretability Founded Monitoring of Medical Image Deep Learning Models

  2024cites this paper
• Explainable artificial intelligence (XAI) to find optimal in-silico biomarkers for cardiac drug toxicity evaluation

  2024cites this paper
• Explainability of CNN-based Alzheimer’s disease detection from online handwriting

  2024cites this paper
• SkipPLUS: Skip the First Few Layers to Better Explain Vision Transformers

  2024cites this paper
• Explainable Earth Surface Forecasting Under Extreme Events

  2024cites this paper
• Seeing Through VisualBERT: A Causal Adventure on Memetic Landscapes

  2024influential citation
• InfoDisent: Explainability of Image Classification Models by Information Disentanglement

  2024cites this paper
• ContextCite: Attributing Model Generation to Context

  2024cites this paper
• Gradient-Free Post-Hoc Explainability Using Distillation Aided Learnable Approach

  2024cites this paper
• Enhancing Federated Learning With Pattern-Based Client Clustering

  2024cites this paper
• PCAIME: Principal Component Analysis-Enhanced Approximate Inverse Model Explanations Through Dimensional Decomposition and Expansion

  2024cites this paper
• Explanation Space: A New Perspective into Time Series Interpretability

  2024cites this paper
• Additive-feature-attribution methods: a review on explainable artificial intelligence for fluid dynamics and heat transfer

  2024cites this paper
• A Learning-based Approach for Explaining Language Models

  2024cites this paper
• Counterfactuals As a Means for Evaluating Faithfulness of Attribution Methods in Autoregressive Language Models

  2024cites this paper
• Exploring Explainable AI Techniques for Radio Frequency Machine Learning

  2024cites this paper
• Do Transformers and CNNs Learn Different Concepts of Brain Age?

  2024cites this paper
• xSiGra: explainable model for single-cell spatial data elucidation

  2024cites this paper
• Your Context Is Not an Array: Unveiling Random Access Limitations in Transformers

  2024cites this paper
• Visualizing Model Behaviors for Clinic Users: Explaining A Clinical Prediction Model for 30-day Readmission after Inpatient Alcohol Dependence Treatment

  2024cites this paper