Can Attention Improve Sequence-to-Point Load Disaggregation? A Comparative Assessment

Adding attention mechanisms to neural networks has been widely explored in recent years. Within the realm of Non-Intrusive Load Monitoring (NILM), however, the potential of attention has not been fully investigated to date. While the few existing works found that attention can lead to lower disaggregation errors and/or greater model robustness, what remains unclear is when and under which conditions these gains can be accomplished. We hence conduct a systematic analysis of different attention mechanisms for energy data disaggregation, i.e., Non-Intrusive Load Monitoring (NILM). More specifically, we select three distinct attention mechanisms: Channel Attention (CA), Feed-Forward Attention (FFA), and Self-Attention (SA) and extend them to construct seven different attention modules, which are integrated into the existing sequence-based neural network architecture to develop enhanced models. We evaluate the performance of the seven configurations on the publicly available UK-DALE and REDD datasets under cross-house and cross-dataset settings to quantify both in-distribution accuracy and transfer to unseen data, as is typical in real-world scenarios. Across these controlled comparisons, we found that the effects of attention are strongly context-dependent. That is, they do not only vary with appliance dynamics, but also depend on the dataset characteristics. While some attention configurations showed notable gains for a few appliances, their use led to a lower accuracy for other devices. What is more, accuracy results also differed when trying to disaggregate the same appliance type in two different datasets. Besides discussing these insights in more depth, we further quantify the resource tradeoffs of each attention family. We show that the incurred overhead (in terms of required computation) differs starkly between attention types, yet does not directly correlate with increased performance. In order to cater for a deeper understanding of the potentials and limitations of attention, our work provides design guidance for both practitioners and model designers on choosing attention families under accuracy and efficiency constraints.

• Publication year

  2025

• Venue

  International Conference on Systems for Energy-Efficient Built Environments

• Publication date

  2025-11-11

• Fields of study

  Computer Science, Engineering, Environmental Science

• Identifiers
  DOI 10.1145/3736425.3772099
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Enhancing Non-Intrusive Load Monitoring Through Transfer Learning with Transformer Models

  2025cited by this paper
• A review of current methods and challenges of advanced deep learning-based non-intrusive load monitoring (NILM) in residential context

  2024cited by this paper
• DP<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" altimg="si116.svg" display="inline" id="d1e562"><mml:msup><mml:mrow /><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msup></mml:math>-NILM: A distributed and privacy-preserving framework for non-intrusive load monitoring

  2024cited by this paper
• Evaluation of the effectiveness of energy sustainability measures through the dynamic energy consumption model

  2024cited by this paper
• NILM-LANN: A Lightweight Attention-based Neural Network in Non-Intrusive Load Monitoring

  2024cited by this paper
• Research on non-intrusive load decomposition model based on parallel multi-scale attention mechanism and its application in smart grid

  2024cited by this paper
• Non-Intrusive Load Monitoring: A Review

  2023cited by this paper
• Attention-Based Multitask Probabilistic Network for Nonintrusive Appliance Load Monitoring

  2023cited by this paper
• An ensemble-policy non-intrusive load monitoring technique based entirely on deep feature-guided attention mechanism

  2022cited by this paper
• Electric energy disaggregation via non-intrusive load monitoring: A state-of-the-art systematic review

  2022cited by this paper
• Improving wireless indoor non-intrusive load disaggregation using attention-based deep learning networks

  2021cited by this paper
• Improving Non-Intrusive Load Disaggregation through an Attention-Based Deep Neural Network

  2021influential reference
• Review on Deep Neural Networks Applied to Low-Frequency NILM

  2021cited by this paper
• A Deep Recurrent Neural Network for Non-Intrusive Load Monitoring Based on Multi-Feature Input Space and Post-Processing

  2020cited by this paper
• BERT4NILM: A Bidirectional Transformer Model for Non-Intrusive Load Monitoring

  2020influential reference
• A critical review of state-of-the-art non-intrusive load monitoring datasets

  2020cited by this paper
• Generative Adversarial Networks and Transfer Learning for Non-Intrusive Load Monitoring in Smart Grids

  2020cited by this paper
• Scale- and Context-Aware Convolutional Non-Intrusive Load Monitoring

  2019cited by this paper
• Transfer Learning for Non-Intrusive Load Monitoring

  2019cited by this paper
• An Attentive Survey of Attention Models

  2019cited by this paper
• Bayesian-optimized Bidirectional LSTM Regression Model for Non-intrusive Load Monitoring

  2019cited by this paper
• A Study on Markovian and Deep Learning Based Architectures for Household Appliance-level Load Modeling and Recognition

  2019cited by this paper
• Deep Learning Application to Non-Intrusive Load Monitoring

  2019cited by this paper
• Denoising autoencoders for Non-Intrusive Load Monitoring: Improvements and comparative evaluation

  2018cited by this paper
• Subtask Gated Networks for Non-Intrusive Load Monitoring

  2018cited by this paper
• Attention is All you Need

  2017influential reference
• Squeeze-and-Excitation Networks

  2017cited by this paper
• Sequence-to-point learning with neural networks for nonintrusive load monitoring

  2016influential reference
• Survey on the attention based RNN model and its applications in computer vision

  2016cited by this paper
• Neural NILM: Deep Neural Networks Applied to Energy Disaggregation

  2015cited by this paper
• Describing Multimedia Content Using Attention-Based Encoder-Decoder Networks

  2015cited by this paper
• Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems

  2015cited by this paper
• Neural Machine Translation by Jointly Learning to Align and Translate

  2014influential reference
• The UK-DALE dataset, domestic appliance-level electricity demand and whole-house demand from five UK homes

  2014cited by this paper
• REDD : A Public Data Set for Energy Disaggregation Research

  2011cited by this paper
• Nonintrusive appliance load monitoring

  1992cited by this paper

• No citing papers are available for this paper.