Interpretable Deep Neural Networks for Single-Trial EEG Classification

BACKGROUND In cognitive neuroscience the potential of deep neural networks (DNNs) for solving complex classification tasks is yet to be fully exploited. The most limiting factor is that DNNs as notorious 'black boxes' do not provide insight into neurophysiological phenomena underlying a decision. Layer-wise relevance propagation (LRP) has been introduced as a novel method to explain individual network decisions. NEW METHOD We propose the application of DNNs with LRP for the first time for EEG data analysis. Through LRP the single-trial DNN decisions are transformed into heatmaps indicating each data point's relevance for the outcome of the decision. RESULTS DNN achieves classification accuracies comparable to those of CSP-LDA. In subjects with low performance subject-to-subject transfer of trained DNNs can improve the results. The single-trial LRP heatmaps reveal neurophysiologically plausible patterns, resembling CSP-derived scalp maps. Critically, while CSP patterns represent class-wise aggregated information, LRP heatmaps pinpoint neural patterns to single time points in single trials. COMPARISON WITH EXISTING METHOD(S) We compare the classification performance of DNNs to that of linear CSP-LDA on two data sets related to motor-imaginary BCI. CONCLUSION We have demonstrated that DNN is a powerful non-linear tool for EEG analysis. With LRP a new quality of high-resolution assessment of neural activity can be reached. LRP is a potential remedy for the lack of interpretability of DNNs that has limited their utility in neuroscientific applications. The extreme specificity of the LRP-derived heatmaps opens up new avenues for investigating neural activity underlying complex perception or decision-related processes.

• Publication year

  2016

• Venue

  Journal of Neuroscience Methods

• Publication date

  2016-04-27

• Fields of study

  Medicine, Computer Science, Mathematics

• Identifiers
  DOI 10.1016/j.jneumeth.2016.10.008arXiv 1604.08201PMID 27746229
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Bringing BCI into everyday life: Motor imagery in a pseudo realistic environment

  2015cited by this paper
• Domain-Adversarial Training of Neural Networks

  2015cited by this paper
• On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation

  2015cited by this paper
• Understanding Neural Networks Through Deep Visualization

  2015cited by this paper
• A Neural Network-Based Optimal Spatial Filter Design Method for Motor Imagery Classification

  2015influential reference
• Analyzing Classifiers: Fisher Vectors and Deep Neural Networks

  2015cited by this paper
• EEG-based classification of video quality perception using steady state visual evoked potentials (SSVEPs)

  2015cited by this paper
• Grand average ERP-image plotting and statistics: A method for comparing variability in event-related single-trial EEG activities across subjects and conditions.

  2015cited by this paper
• On the use of convolutional neural networks and augmented CSP features for multi-class motor imagery of EEG signals classification

  2015cited by this paper
• Evaluating the Visualization of What a Deep Neural Network Has Learned

  2015cited by this paper
• Machine learning of molecular electronic properties in chemical compound space

  2013cited by this paper
• Deep learning for neuroimaging: a validation study

  2013cited by this paper
• Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank

  2013cited by this paper
• Single-trial analysis of the neural correlates of speech quality perception

  2013cited by this paper
• ImageNet classification with deep convolutional neural networks

  2012cited by this paper
• The timing of exploratory decision-making revealed by single-trial topographic EEGanalyses

  2012cited by this paper
• Efficient BackProp

  2012cited by this paper
• Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis

  2011cited by this paper
• Single-trial analysis and classification of ERP components - A tutorial

  2011cited by this paper
• Author ' s personal copy A Fast Quartet tree heuristic for hierarchical clustering

  2010cited by this paper
• 2009 Special Issue: Subject-independent mental state classification in single trials

  2009cited by this paper
• Optimizing Spatial filters for Robust EEG Single-Trial Analysis

  2008cited by this paper
• The BCI competition III: validating alternative approaches to actual BCI problems

  2006cited by this paper
• Recipes for the linear analysis of EEG

  2005cited by this paper
• Neural Networks: Tricks of the Trade

  2002cited by this paper

• Application of Explainable AI in Neuroscience: Enhancing Autism Screening

  2026cites this paper
• G3DCT: An Interpretable Spatial Grid-based Framework with Temporal Convolution-Transformer for EEG Artifact Identification

  2026cites this paper
• TVGNet: A novel hybrid EEG motor imagery decoding network

  2026cites this paper
• Explainable Artificial Intelligence (XAI) for EEG Analysis: A Survey on Recent Trends and Advancements

  2026cites this paper
• From epilepsy seizure classification to detection: A deep learning-based approach for raw EEG signals

  2026cites this paper
• Unlocking Transcranial FUS-EEG Feature Fusion for Non-Invasive Sleep Staging in Next-Gen Clinical Applications

  2025cites this paper
• Enhanced EEG signal classification in brain computer interfaces using hybrid deep learning models

  2025cites this paper
• The role of electroencephalography in epilepsy research—From seizures to interictal activity and comorbidities

  2025cites this paper
• Explainable AI in EEG Waves Based Classification for Early Identification in Autism

  2025cites this paper
• Improving Motor Imagery EEG Signal Classification Using Deep Learning Techniques

  2025cites this paper
• Decision-making components and times revealed by the single-trial electroencephalogram

  2025cites this paper
• EEG-Based Emotion Recognition: A Review and Emerging Paths

  2025cites this paper
• Explainable AI for Sensor Signal Interpretation to Revolutionize Human Health Monitoring: A Review

  2025influential citation
• Noise-Robust and Interpretable EEG Classification of Hyperactive Disorders Using Graph Convolutional Network

  2025cites this paper
• Explainable artificial intelligence-based identification of the localized events in imagined speech electroencephalogram

  2025cites this paper
• Decoding brain signals: A comprehensive review of EEG-Based BCI paradigms, signal processing and applications

  2025cites this paper
• Hybrid Deep Learning Models for Accurate EEG-Based Cognitive State Classification

  2025cites this paper
• Geo-GCN: Geometric-Graphical Convolutional Network for EEG-based Auditory Attention Detection

  2025cites this paper
• EffortNet: A Deep Learning Framework for Objective Assessment of Speech Enhancement Technologies Using EEG-Based Alpha Oscillations

  2025cites this paper
• Foundation Models for Neural Signal Decoding: EEG‐Centered Perspectives Toward Unified Representations

  2025cites this paper
• Deep Learning Meets Source Reconstruction

  2025cites this paper
• Brain-machine interaction and assist effect evaluation of a single-degree-of-freedom sit-to-stand transfer robot

  2025cites this paper
• Intuitive, phenomenological approach to test methods for approval purposes of trained models using the example of predicting loudness classes from EEG-data

  2025cites this paper
• Deep Learning From Diffuse Optical Oximetry Time-Series: An fNIRS-Focused Review of Recent Advancements and Future Directions

  2025cites this paper
• Explainable machine learning on weighted connectivity networks across frequencies for outcome prediction in comatose patients

  2025cites this paper
• The application of electroencephalogram in depression research: bibliometric and technological application analysis from 2005 to 2025

  2025cites this paper
• Non-invasive Brain Stimulation-Based Sleep Stage Classification Using Transcranial Infrared Based Electrocardiogram

  2025cites this paper
• PBFE-DAN:Personal Biological Feature Enhanced Domain Adaptation Network for Cross-Session Brainprint Recognition

  2025cites this paper
• Empowering surgeons: will artificial intelligence change oral and maxillofacial surgery?

  2024cites this paper
• Real-Time Calculation of CO2 Conversion in Radio-Frequency Discharges under Martian Pressure by Introducing Deep Neural Network

  2024cites this paper
• EEG Emotion Copilot: Pruning LLMs for Emotional EEG Interpretation with Assisted Medical Record Generation

  2024cites this paper
• The Role of Explainable AI in Revolutionizing Human Health Monitoring

  2024cites this paper
• From Epilepsy Seizures Classification to Detection: A Deep Learning-based Approach for Raw EEG Signals

  2024cites this paper
• MSMGE-CNN: a multi-scale multi-graph embedding convolutional neural network for motor related EEG decoding

  2024cites this paper
• A Holistic review and performance evaluation of unsupervised learning methods for network anomaly detection

  2024cites this paper
• A Novel Approach of Brain-Computer Interfacing (BCI) and Grad-CAM Based Explainable Artificial Intelligence: Use Case Scenario for Smart Healthcare.

  2024cites this paper
• WearNeuroNet: An Interpretable Light-Weight Deep Learning Approach for Ictal–Interictal Classification for Limited Channel EEG Wearables

  2024cites this paper
• Explaining the predictions of kernel SVM models for neuroimaging data analysis

  2024cites this paper
• Exploration of an intrinsically explainable self-attention based model for prototype generation on single-channel EEG sleep stage classification

  2024cites this paper
• Unraveling motor imagery brain patterns using explainable artificial intelligence based on Shapley values

  2024cites this paper
• Flexible Dry Electrode Based on a Wrinkled Surface That Uses Carbon Nanotube/Polymer Composites for Recording Electroencephalograms

  2024cites this paper
• A new method applied for explaining the landing patterns: Interpretability analysis of machine learning

  2024cites this paper
• Convolutional neural networks reveal properties of reach-to-grasp encoding in posterior parietal cortex

  2024cites this paper
• XAI-Driven sEMG Feature Analysis for Hand Gestures

  2024cites this paper
• Multi-branch spatial-temporal-spectral convolutional neural networks for multi-task motor imagery EEG classification

  2024cites this paper
• The Role of Explainable AI in Revolutionizing Human Health Monitoring: A Review

  2024cites this paper
• Formulation of Common Spatial Patterns for Multi-Task Hyperscanning BCI

  2024cites this paper
• Physiological signal analysis using explainable artificial intelligence: A systematic review

  2024cites this paper
• Deep Learning made transferable: studying Brain decoding

  2024cites this paper
• Improved data quality and statistical power of trial-level event-related potentials with Bayesian random-shift Gaussian processes

  2024cites this paper
• Identifying EEG Biomarkers of Depression with Novel Explainable Deep Learning Architectures

  2024cites this paper
• Research on EEG Signal Classification Method Based on Brain-Computer Interface Control of Rehabilitation Robot

  2024cites this paper
• EEG Emotion Copilot: Optimizing lightweight LLMs for emotional EEG interpretation with assisted medical record generation

  2024cites this paper
• Methodological Explainability Evaluation of an Interpretable Deep Learning Model for Post-Hepatectomy Liver Failure Prediction Incorporating Counterfactual Explanations and Layerwise Relevance Propagation: A Prospective In Silico Trial

  2024cites this paper
• Explainable AI for Time Series via Virtual Inspection Layers

  2023influential citation
• Deep neural networks and gradient-weighted class activation mapping to classify and analyze EEG

  2023cites this paper
• Pairing explainable deep learning classification with clustering to uncover effects of schizophrenia upon whole brain functional network connectivity dynamics

  2023cites this paper
• Deep comparisons of Neural Networks from the EEGNet family

  2023cites this paper
• Impartial competitive learning in multi-layered neural networks

  2023cites this paper
• Designing an XAI interface for BCI experts: A contextual design for pragmatic explanation interface based on domain knowledge in a specific context

  2023cites this paper
• Advancement in Machine Learning: A Strategic Lookout from Cancer Identification to Treatment

  2023cites this paper
• A remote and personalised novel approach for monitoring asthma severity levels from EEG signals utilizing classification algorithms

  2023cites this paper
• EEG-Based Emotion Recognition via Knowledge-Integrated Interpretable Method

  2023cites this paper
• Dysarthria detection based on a deep learning model with a clinically-interpretable layer

  2023cites this paper
• Machine learning in biosignals processing for mental health: A narrative review

  2023cites this paper
• Review of Machine Learning and Artificial Intelligence (ML/AI) for the Pediatric Neurologist.

  2023cites this paper
• FATE in AI: Towards Algorithmic Inclusivity and Accessibility

  2023cites this paper
• Reduce System Redundancy and Optimize Sensor Disposition for EMG–IMU Multimodal Fusion Human–Machine Interfaces With XAI

  2023cites this paper
• Characterization of attentional event-related potential from REM sleep behavior disorder patients based on explainable machine learning

  2023cites this paper
• Counterfactual Explanation and Instance-Generation using Cycle-Consistent Generative Adversarial Networks

  2023cites this paper
• A survey on XAI and natural language explanations

  2023cites this paper
• Explaining Convolutional Neural Networks for EEG-based Brain-Computer Interface Using Influence Functions

  2023cites this paper
• An in-depth survey on Deep Learning-based Motor Imagery Electroencephalogram (EEG) classification

  2023cites this paper
• Electroencephalogram Analysis Method to Detect Unspoken Answers to Questions Using Multistage Neural Networks

  2023influential citation
• Explainable artificial intelligence approaches for brain–computer interfaces: a review and design space

  2023cites this paper
• Spatio-Temporal Explanation of 3D-EEGNet for Motor Imagery EEG Classification Using Permutation and Saliency

  2023cites this paper
• Explaining ANN-modeled fMRI Data with Path-Weights and Layer-Wise Relevance Propagation

  2023cites this paper
• Contradiction neutralization for interpreting multi-layered neural networks

  2023cites this paper
• Decoding movement kinematics from EEG using an interpretable convolutional neural network

  2023cites this paper
• Using Artificial Intelligence to Predict and Prevent Future Food Insecurity

  2023cites this paper
• A Human-Centric Metaverse Enabled by Brain-Computer Interface: A Survey

  2023cites this paper
• The effects of layer-wise relevance propagation-based feature selection for EEG classification: a comparative study on multiple datasets

  2023cites this paper
• SIGNed explanations: Unveiling relevant features by reducing bias

  2023cites this paper
• Deep learning applied to EEG source-data reveals both ventral and dorsal visual stream involvement in holistic processing of social stimuli

  2023cites this paper
• An empirical comparison of deep learning explainability approaches for EEG using simulated ground truth

  2023cites this paper
• ImageLM: Interpretable image-based learner modelling for classifying learners' computational thinking

  2023cites this paper
• Electroencephalography Signal Processing: A Comprehensive Review and Analysis of Methods and Techniques

  2023cites this paper
• Visualization of Deep Convolutional Neural Networks to Investigate Porous Nanocomposites for Electromagnetic Interference Shielding.

  2023cites this paper
• Interpreting Deep Learning Models for Multi-modal Neuroimaging

  2023cites this paper
• Interpretable and Robust AI in EEG Systems: A Survey

  2023cites this paper
• A Framework for Systematically Evaluating the Representations Learned by A Deep Learning Classifier from Raw Multi-Channel Electroencephalogram Data

  2023cites this paper
• An Interpretable and Attention-based Method for Gaze Estimation Using Electroencephalography

  2023cites this paper
• Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspaces

  2022cites this paper
• Explainable AI: A review of applications to neuroimaging data

  2022cites this paper
• Novel methods for elucidating modality importance in multimodal electrophysiology classifiers

  2022cites this paper
• Relevance-based channel selection in motor imagery brain–computer interface

  2022cites this paper
• Motor Imagery Classification for Brain Computer Interface Using Deep Convolutional Neural Networks and Mixup Augmentation

  2022cites this paper
• Analysis of a Deep Learning Model for 12-Lead ECG Classification Reveals Learned Features Similar to Diagnostic Criteria

  2022cites this paper
• Explainable Machine-Learning-Based Characterization of Abnormal Cortical Activities for Working Memory of Restless Legs Syndrome Patients

  2022cites this paper
• Occult mathematics under the learning process in convolutional neural networks

  2022cites this paper