Intriguing properties of neural networks

Deep neural networks are highly expressive models that have recently achieved state of the art performance on speech and visual recognition tasks. While their expressiveness is the reason they succeed, it also causes them to learn uninterpretable solutions that could have counter-intuitive properties. In this paper we report two such properties. First, we find that there is no distinction between individual high level units and random linear combinations of high level units, according to various methods of unit analysis. It suggests that it is the space, rather than the individual units, that contains of the semantic information in the high layers of neural networks. Second, we find that deep neural networks learn input-output mappings that are fairly discontinuous to a significant extend. We can cause the network to misclassify an image by applying a certain imperceptible perturbation, which is found by maximizing the network's prediction error. In addition, the specific nature of these perturbations is not a random artifact of learning: the same perturbation can cause a different network, that was trained on a different subset of the dataset, to misclassify the same input.

• Publication year

  2013

• Venue

  International Conference on Learning Representations

• Publication date

  2013-12-20

• Fields of study

  Computer Science

• Identifiers
  arXiv 1312.6199
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Visualizing and Understanding Convolutional Neural Networks

  2013influential reference
• Visualizing and Understanding Convolutional Networks

  2013cited by this paper
• Efficient Estimation of Word Representations in Vector Space

  2013cited by this paper
• Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

  2013cited by this paper
• Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups

  2012cited by this paper
• ImageNet classification with deep convolutional neural networks

  2012influential reference
• Building high-level features using large scale unsupervised learning

  2011cited by this paper
• Measuring Invariances in Deep Networks

  2009cited by this paper
• How to Explain Individual Classification Decisions

  2009influential reference
• ImageNet: A large-scale hierarchical image database

  2009cited by this paper
• Visualizing Higher-Layer Features of a Deep Network

  2009cited by this paper
• A discriminatively trained, multiscale, deformable part model

  2008cited by this paper
• Learning Deep Architectures for AI

  2007cited by this paper
• The mnist database of handwritten digits

  2005cited by this paper

• Adversarial Shadows in Digital Forensics: New Insights Into File Fragment Classification Vulnerabilities and Defenses

  2026cites this paper
• A Multi-Stage Backdoor Detection (MSBD) Framework

  2026cites this paper
• Audio Adversarial Example With No Noise in the Silent Area for Speech Recognition System

  2026cites this paper
• Noisy Analysis of Quantum SMOTE on Condition Monitoring and Fault Classification in Industrial and Energy Systems

  2026cites this paper
• ARMOR: Agentic Reasoning for Methods Orchestration and Reparameterization for Robust Adversarial Attacks

  2026cites this paper
• Operator ℓ∞→ℓ∞ norm of products of random matrices with iid entries

  2026cites this paper
• ALDA: Enhancing the transferability of adversarial attacks with attention-guided look-ahead and data augmentation

  2026cites this paper
• Unsharp-Inspired Adversarial Point Cloud Perturbation via Low-Rank Approximation

  2026cites this paper
• A survey on adversarial machine learning: Attacks, defenses, real-world applications, and future research directions

  2026cites this paper
• Building Production-Ready Probes For Gemini

  2026cites this paper
• On the Effects of Adversarial Perturbations on Distribution Robustness

  2026cites this paper
• Adversarially robust neural network decision boundaries via tropical geometry.

  2026cites this paper
• Perturbation-Induced Linearization: Constructing Unlearnable Data with Solely Linear Classifiers

  2026cites this paper
• Structured Matrix Constraint Systems for Architecture-Hiding Succinct Zero-Knowledge Proofs for Neural Networks

  2026cites this paper
• ADMM-Based Adversarial False Data Injection Attacks Against Multi-Label Locational Detection

  2026cites this paper
• Adversarial image detection based on spatial and frequency information

  2026cites this paper
• Toward Certifiably Robust Face Recognition: Analyses and Improvements

  2026cites this paper
• Adversarial attack-defense framework for enhancing the robustness of power insulator detection in cloud-edge deployment

  2026cites this paper
• Time-constrained adversarial attacks for video recognition models: temporally sparse but effective perturbations

  2026cites this paper
• SHIELD: Semantic-guided graph contrastive learning for malware detection

  2026cites this paper
• Hierarchical Refinement of Universal Multimodal Attacks on Vision-Language Models

  2026influential citation
• Securing DNN Acceleration From Off-Chip Memory Vulnerabilities With Low-Overhead Authenticated Encryption

  2026cites this paper
• DDSA: Dual-Domain Strategic Attack for Spatial-Temporal Efficiency in Adversarial Robustness Testing

  2026cites this paper
• A comprehensive analysis of objective functions in adversarial training

  2026cites this paper
• Persona Jailbreaking in Large Language Models

  2026cites this paper
• CAA: Toward Camouflaged and Transferable Adversarial Examples

  2026cites this paper
• LipNeXt: Scaling up Lipschitz-based Certified Robustness to Billion-parameter Models

  2026cites this paper
• Dynamic Mask-Based Backdoor Attack Against Vision AI Models: A Case Study on Mushroom Detection

  2026cites this paper
• BadDet+: Robust Backdoor Attacks for Object Detection

  2026cites this paper
• LAMP: Learning Universal Adversarial Perturbations for Multi-Image Tasks via Pre-trained Models

  2026cites this paper
• Semantic Communication-Based Aerial–Maritime Energy Trade-Off for Maritime Mobile Edge Computing Networks Under Jamming

  2026cites this paper
• CCIFE: Channel-Resilient Ensemble Adversarial Attack Against DNN-Based Modulation Classifiers

  2026cites this paper
• Identity-Preserving Covert Communication With Generative Perturbation

  2026cites this paper
• PrIdentity: Generalizable Privacy-Preserving Adversarial Perturbations for Anonymizing Facial Identity

  2026cites this paper
• Inverse machine learning for the design of perforated beams: Parent section and material prediction

  2026cites this paper
• S3AT: Self-Paced, Self-Distilled, and Self-Finetuned Adversarial Training for Robust Automatic Modulation Recognition

  2026cites this paper
• A survey on physical adversarial attacks against face recognition systems

  2026cites this paper
• Reinforcing Adversarial Transferability via Negative Class Guided Example Generation

  2026cites this paper
• NADD: Amplifying Noise for Effective Diffusion-based Adversarial Purification

  2026cites this paper
• Aligned explanations in neural networks

  2026cites this paper
• Low-visibility adversarial sample generation method based on human visual perception

  2026influential citation
• Improving Flat Maxima with Natural Gradient for Better Adversarial Transferability

  2026cites this paper
• LDLT L-Lipschitz Network Weight Parameterization Initialization

  2026cites this paper
• Diffusion-Driven Deceptive Patches: Adversarial Manipulation and Forensic Detection in Facial Identity Verification

  2026cites this paper
• Adversarial Vulnerability from On-Manifold Inseparability and Poor Off-Manifold Convergence

  2026cites this paper
• A Reconstruction-Based Defense Framework for Automatic Modulation Recognition

  2026cites this paper
• Adversarial News and Lost Profits: Manipulating Headlines in LLM-Driven Algorithmic Trading

  2026cites this paper
• How Worst-Case Are Adversarial Attacks? Linking Adversarial and Perturbation Robustness

  2026influential citation
• Dialectal substitution as an adversarial approach for evaluating Arabic NLP robustness

  2026cites this paper
• BLTOO-MFFL: Automated and Adversarially Robust Deep Learning for SAR Image Classification by Bi-Level Three-Objective Optimization and Multi-Feature Fusion Loss

  2026cites this paper
• Analyzing Neural Network Information Flow Using Differential Geometry

  2026cites this paper
• SoundBreak: A Systematic Study of Audio-Only Adversarial Attacks on Trimodal Models

  2026cites this paper
• HGA: Heuristic Black‐Box Test Case Generation for NLP Intelligent Software

  2026cites this paper
• Towards a robust adversarial patch attack against RGB-T crowd counting

  2026cites this paper
• Robust Privacy: Inference-Time Privacy through Certified Robustness

  2026cites this paper
• Information Hidden in Gradients of Regression with Target Noise

  2026cites this paper
• OTI: A Model-free and Visually Interpretable Measure of Image Attackability

  2026cites this paper
• GlassCurtainCrackAdversarialDefense: A convolutional and attention-based adversarial defense network for glass curtain wall crack detection

  2026cites this paper
• Generalized Transferable Attack Across Datasets

  2026cites this paper
• Toward Robust Agents: A Survey of Adversarial Attacks and Defenses in Deep Reinforcement Learning

  2026cites this paper
• Adaptively Robust Resettable Streaming

  2026cites this paper
• XFACTORS: Disentangled Information Bottleneck via Contrastive Supervision

  2026cites this paper
• Rectifying Adversarial Examples Using Their Vulnerabilities

  2026cites this paper
• Using Adversarial Training to Improve Uncertainty Quantification

  2026cites this paper
• Robustness Assessment of DL-Based Automatic Modulation Classification Model via Channel-Aware Adversarial Perturbation

  2026cites this paper
• A Survey of Wireless Sensing Security From a Role-Based View

  2026cites this paper
• Learning Universal Attack via Model-Guided Meta-Learning for Person Reidentification

  2026cites this paper
• In-Situ Metrology for Roll-to-Roll Microcontact Printing via Condensation Figures and YOLOv8

  2026cites this paper
• Adversarial Transfer Attack Against and Adaptive Defense for Intelligent Modulation Recognition

  2026cites this paper
• Evaluating the Adversarial Robustness of Vision-Language Models via Internal Feature Perturbations

  2026cites this paper
• Adversarial Attack Resilient Computational Modeling for Person Re-Identification in Visual IoT Applications

  2026cites this paper
• Fast and Effective Overwrite Attack Against DNN-Based Image Watermarking Models

  2026cites this paper
• Global aggregated gradient-guided adversarial attacks for person re-identification

  2026cites this paper
• Analyzing Fairness of Neural Network Prediction via Counterfactual Dataset Generation

  2026cites this paper
• Adversarial Attack Priors-Based Nondivergence Diffusion Equation Model for SAR Speckle Removal

  2026cites this paper
• CoGA: A Collaborative Gray-Box Adversarial Attack for Multimodal Language Models

  2026cites this paper
• Towards Patch-Based Noise Compression for Adversarial Attack Against Transformer-Based Visual Tracking

  2026cites this paper
• Exploiting Shared Adversarial Features for Dynamic Attacks in Large Vision-Language Models

  2026cites this paper
• IO-RAE: Information-Obfuscation Reversible Adversarial Example for Audio Privacy Protection

  2026cites this paper
• Deep Robust Koopman Learning from Noisy Data

  2026cites this paper
• PartImageNet++ Dataset: Enhancing Visual Models with High-Quality Part Annotations

  2026cites this paper
• Detecting Semantic Backdoors in a Mystery Shopping Scenario

  2026cites this paper
• Bayesian deep learning for probabilistic aquifer vulnerability and uncertainty prediction

  2026cites this paper
• Fahrzeugsicherheit autonomer Fahrzeuge: Grenzen heutiger Systeme und potenzielle Lösungen

  2026cites this paper
• QHNEAD-Quantum Hyperdimensional Neuro Symbolic Evolving Adversarial Defense

  2026cites this paper
• FST: Improving adversarial robustness via feature similarity-based targeted adversarial training

  2026cites this paper
• Image Representation Induced Subspaces for Practical Classification Robustness

  2026cites this paper
• Baiting AI: Deceptive Adversary Against AI-Protected Industrial Infrastructures

  2026cites this paper
• Efficient State Preparation for Quantum Machine Learning

  2026cites this paper
• Adversarial Attacks on Evolutionary Algorithms Solving Data-Driven Optimization Problems [Research Frontier]

  2026cites this paper
• Adversarial Evasion Attacks on Computer Vision using SHAP Values

  2026cites this paper
• SRAW-Attack: Space-Reweighted Adversarial Warping Attack for SAR Target Recognition

  2026cites this paper
• Formal Methods in Robot Policy Learning and Verification: A Survey on Current Techniques and Future Directions

  2026cites this paper
• Overcoming Open-Set Approaches to Adversarial Defense

  2026cites this paper
• Proxy Robustness in Vision Language Models is Effortlessly Transferable

  2026cites this paper
• Feature-Aware Test Generation for Deep Learning Models

  2026cites this paper
• Orthogonium : A Unified, Efficient Library of Orthogonal and 1-Lipschitz Building Blocks

  2026cites this paper
• Towards Robust Universal Perturbation Attacks: A Float-Coded, Penalty-Driven Evolutionary Approach

  2026influential citation
• On damage of interpolation to adversarial robustness in regression

  2026cites this paper
• Robust Machine-vision models against adversarial attacks in Industry 4.0

  2026cites this paper