The Confusion is Real: GRAPHIC - A Network Science Approach to Confusion Matrices in Deep Learning

Explainable artificial intelligence has emerged as a promising field of research to address reliability concerns in artificial intelligence. Despite significant progress in explainable artificial intelligence, few methods provide a systematic way to visualize and understand how classes are confused and how their relationships evolve as training progresses. In this work, we present GRAPHIC, an architecture-agnostic approach that analyzes neural networks on a class level. It leverages confusion matrices derived from intermediate layers using linear classifiers. We interpret these as adjacency matrices of directed graphs, allowing tools from network science to visualize and quantify learning dynamics across training epochs and intermediate layers. GRAPHIC provides insights into linear class separability, dataset issues, and architectural behavior, revealing, for example, similarities between flatfish and man and labeling ambiguities validated in a human study. In summary, by uncovering real confusions, GRAPHIC offers new perspectives on how neural networks learn. The code is available at https://github.com/Johanna-S-Froehlich/GRAPHIC.

• Publication year

  2026

• Venue

  Unknown venue

• Publication date

  2026-02-23

• Fields of study

  Computer Science

• Identifiers
  arXiv 2602.19770
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Demystifying the Accuracy-Interpretability Trade-Off: A Case Study of Inferring Ratings from Reviews

  2025cited by this paper
• Explainable artificial intelligence (XAI): from inherent explainability to large language models

  2025cited by this paper
• Explainable artificial intelligence: A survey of needs, techniques, applications, and future direction

  2024cited by this paper
• A Survey on Convolutional Neural Networks and Their Performance Limitations in Image Recognition Tasks

  2024cited by this paper
• The geometry of hidden representations of large transformer models

  2023cited by this paper
• Unsupervised visualization of image datasets using contrastive learning

  2022cited by this paper
• On the inadequacy of nominal assortativity for assessing homophily in networks

  2022cited by this paper
• Training a Vision Transformer from scratch in less than 24 hours with 1 GPU

  2022cited by this paper
• Identifying accurate link predictors based on assortativity of complex networks

  2022cited by this paper
• Concept Activation Regions: A Generalized Framework For Concept-Based Explanations

  2022cited by this paper
• Attention-based Interpretability with Concept Transformers

  2022cited by this paper
• Optimizing Relevance Maps of Vision Transformers Improves Robustness

  2022cited by this paper
• Post-hoc Concept Bottleneck Models

  2022cited by this paper
• From the origin of life to pandemics: emergent phenomena in complex systems

  2022cited by this paper
• A Simple Episodic Linear Probe Improves Visual Recognition in the Wild

  2022cited by this paper
• "Why Not Other Classes?": Towards Class-Contrastive Back-Propagation Explanations

  2022cited by this paper
• Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks

  2021cited by this paper
• Do Vision Transformers See Like Convolutional Neural Networks?

  2021influential reference
• Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations

  2021cited by this paper
• Transparency and the Black Box Problem: Why We Do Not Trust AI

  2021cited by this paper
• Visualization of the Feature Space of Neural Networks

  2020cited by this paper
• A Survey on Neural Network Interpretability

  2020cited by this paper
• Concept Bottleneck Models

  2020cited by this paper
• Data Compression and Visualization Using PCA and T-SNE

  2020cited by this paper
• Explainable Artificial Intelligence for 6G: Improving Trust between Human and Machine

  2020cited by this paper
• ConfusionFlow: A Model-Agnostic Visualization for Temporal Analysis of Classifier Confusion

  2019cited by this paper
• Interpreting Intentionally Flawed Models with Linear Probes

  2019cited by this paper
• Characterizing the Decision Boundary of Deep Neural Networks

  2019cited by this paper
• Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges

  2019cited by this paper
• Similarity of Neural Network Representations Revisited

  2019cited by this paper
• Intrinsic dimension of data representations in deep neural networks

  2019cited by this paper
• UMAP: Uniform Manifold Approximation and Projection

  2018cited by this paper
• T-SNE-CUDA: GPU-Accelerated T-SNE and its Applications to Modern Data

  2018cited by this paper
• Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead

  2018cited by this paper
• Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)

  2017cited by this paper
• Network science of biological systems at different scales: A review.

  2017cited by this paper
• Confusion Graph: Detecting Confusion Communities in Large Scale Image Classification

  2017influential reference
• Stock market as temporal network

  2017cited by this paper
• Do Convolutional Neural Networks Learn Class Hierarchy?

  2017cited by this paper
• Decoupled Weight Decay Regularization

  2017influential reference
• Interpretable Decision Sets: A Joint Framework for Description and Prediction

  2016cited by this paper
• Understanding intermediate layers using linear classifier probes

  2016influential reference
• How to Use t-SNE Effectively

  2016cited by this paper
• BranchyNet: Fast inference via early exiting from deep neural networks

  2016cited by this paper
• Visualizing Large-scale and High-dimensional Data

  2016cited by this paper
• Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization

  2016cited by this paper
• “Why Should I Trust You?”: Explaining the Predictions of Any Classifier

  2016cited by this paper
• Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model

  2015cited by this paper
• Tiny ImageNet Visual Recognition Challenge

  2015influential reference
• Deep Residual Learning for Image Recognition

  2015influential reference
• Distilling the Knowledge in a Neural Network

  2015cited by this paper
• Directed Louvain : maximizing modularity in directed networks

  2015cited by this paper
• On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation

  2015cited by this paper
• HD-CNN: Hierarchical Deep Convolutional Neural Networks for Large Scale Visual Recognition

  2014cited by this paper
• Gephi: An Open Source Software for Exploring and Manipulating Networks

  2009cited by this paper
• Learning Multiple Layers of Features from Tiny Images

  2009influential reference
• Visualizing Data using t-SNE

  2008cited by this paper
• Community structure in directed networks.

  2007cited by this paper
• The scaling laws of human travel

  2006cited by this paper
• Analysis of weighted networks.

  2004influential reference
• The Structure and Function of Complex Networks

  2003influential reference
• Mixing patterns in networks.

  2002influential reference
• Exploring complex networks

  2001cited by this paper
• On the Surprising Behavior of Distance Metrics in High Dimensional Spaces

  2001cited by this paper
• A Simple Weight Decay Can Improve Generalization

  1991cited by this paper

• No citing papers are available for this paper.