A novel parametric scaled exponential linear unit activation function for deep residual networks in short-term load forecasting

Short-Term Load Forecasting (STLF) is essential for ensuring the stability and operational efficiency of modern power systems. Deep Residual Networks (DRNs) have recently demonstrated promising performance in this domain, enabling more effective training of deeper architectures. However, existing activation functions such as the Scaled Exponential Linear Unit (SELU) rely on fixed parameters and strict initialization, which may limit their adaptability to seasonally varying load–weather conditions. This study introduces a modified activation function, the Parametric Scaled Exponential Linear Unit (PSELU), which incorporates a small tunable parameter γ in the negative region, extending the formulation of SELU while preserving its self-normalizing characteristics. Experiments conducted within the DRN framework on two benchmark datasets—ISO-NE (temperate climate) and Malaysia (tropical climate)—demonstrate that the DRN model employing the proposed PSELU (γ = 0.02) achieves modest yet consistent improvements in forecasting accuracy compared with the DRN model using SELU. Specifically, the Mean Absolute Percentage Error (MAPE) decreased from 1.718% to 1.662% for ISO-NE and from 5.251% to 5.012% for Malaysia. Although the improvements are moderate, they were statistically validated through 10,000-iteration Bootstrap resampling at the 95% confidence level. These results suggest that the limited parameterization of SELU enhances consistency and adaptability in forecasting performance across different climatic and seasonal conditions. Future work will expand the evaluation to a wider range of datasets and model architectures to further examine the generalizability and practical applicability of PSELU in diverse forecasting contexts.

• Publication year

  2026

• Venue

  Scientific Reports

• Publication date

  2026-01-07

• Fields of study

  Medicine, Computer Science, Engineering

• Identifiers
  DOI 10.1038/s41598-025-32564-zPMID 41495229PMCID 12820360
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• A novel parametric scaled exponential linear unit activation function for deep residual networks in short-term load forecasting

  2026cited by this paper
• RoSwish: A novel Rotating Swish activation function with adaptive rotation around zero

  2025cited by this paper
• Power Grid Load Forecasting Using a CNN-LSTM Network Based on a Multi-Modal Attention Mechanism

  2025influential reference
• TDCN: A novel temporal depthwise convolutional network for short-term load forecasting

  2025cited by this paper
• TXtreme: transformer-based extreme value prediction framework for time series forecasting

  2025cited by this paper
• A multi-stage ensemble model for power load forecasting based on decomposition, error factors, and multi-objective optimization algorithm

  2024cited by this paper
• Simple yet effective adaptive activation functions for physics-informed neural networks

  2024cited by this paper
• Kolmogorov-Arnold Networks (KANs) for Time Series Analysis

  2024cited by this paper
• Adaptive activation functions for predictive modeling with sparse experimental data

  2024cited by this paper
• Short-Term Load Forecasting Utilizing a Combination Model: A Brief Review

  2024cited by this paper
• A comprehensive review on deep learning approaches for short-term load forecasting

  2024cited by this paper
• Short-term load forecasting based on CEEMDAN and Transformer

  2023cited by this paper
• Using weather data in energy time series forecasting: the benefit of input data transformations

  2023cited by this paper
• TS2ARCformer: A Multi-Dimensional Time Series Forecasting Framework for Short-Term Load Prediction

  2023cited by this paper
• Residual LSTM based short-term load forecasting

  2023cited by this paper
• A Snapshot-Stacked Ensemble and Optimization Approach for Vehicle Breakdown Prediction

  2023cited by this paper
• An improved encoder-decoder-based CNN model for probabilistic short-term load and PV forecasting

  2023cited by this paper
• Short-term load forecasting for industrial users based on Transformer-LSTM hybrid model

  2022influential reference
• Deep ResNet-Based Ensemble Model for Short-Term Load Forecasting in Protection System of Smart Grid

  2022cited by this paper
• Short-Term Probability Density Function Forecasting of Industrial Loads Based on ConvLSTM-MDN

  2022cited by this paper
• A Transformer Based Method with Wide Attention Range for Enhanced Short-term Load Forecasting

  2022cited by this paper
• A modified deep residual network for short-term load forecasting

  2022cited by this paper
• Integration of Ensemble GoogLeNet and Modified Deep Residual Networks for Short-Term Load Forecasting

  2021influential reference
• Evolution of Activation Functions: An Empirical Investigation

  2021cited by this paper
• Short-term Load Forecasting for Distribution Substations Based on Residual Neutral Networks and Long Short-Term Memory Neutral Networks with Attention Mechanism

  2021cited by this paper
• Short-Term Load Forecasting With Deep Learning

  2021cited by this paper
• Convolutional residual network to short-term load forecasting

  2020cited by this paper
• Short-Term Load Forecasting Based on Deep Neural Networks Using LSTM Layer

  2020cited by this paper
• Short-term load forecasting scheme based on improved deep residual network and LSTM

  2020cited by this paper
• Discovering Parametric Activation Functions

  2020cited by this paper
• Ensemble Residual Networks for Short-Term Load Forecasting

  2020cited by this paper
• A bootstrap approach is a superior statistical method for the comparison of non-normal data with differing variances.

  2020cited by this paper
• Application of Bidirectional Recurrent Neural Network Combined With Deep Belief Network in Short-Term Load Forecasting

  2019cited by this paper
• Deep Learning for Household Load Forecasting—A Novel Pooling Deep RNN

  2018cited by this paper
• Short-Term Load Forecasting With Deep Residual Networks

  2018influential reference
• Short-Term Load Forecasting using optimized LSTM Networks via Improved Bat Algorithm

  2018cited by this paper
• Long short term memory networks for short-term electric load forecasting

  2017cited by this paper
• Self-Normalizing Neural Networks

  2017influential reference
• Snapshot Ensembles: Train 1, get M for free

  2017cited by this paper
• Improving Economic Values of Day-Ahead Load Forecasts to Real-Time Power System Operations

  2017cited by this paper
• Everything is Image: CNN-based Short-Term Electrical Load Forecasting for Smart Grid

  2017cited by this paper
• Electrical load forecasting models: A critical systematic review

  2017cited by this paper
• A Novel RBF Training Algorithm for Short-Term Electric Load Forecasting and Comparative Studies

  2015cited by this paper
• A Strategy for Short-Term Load Forecasting by Support Vector Regression Machines

  2013cited by this paper
• Rectifier Nonlinearities Improve Neural Network Acoustic Models

  2013cited by this paper
• Short-term load forecasting: Multi-level wavelet neural networks with holiday corrections

  2009cited by this paper
• Neural networks for short-term load forecasting: a review and evaluation

  2001cited by this paper
• Analysis of the value for unit commitment of improved load forecasts

  1999cited by this paper

• A novel parametric scaled exponential linear unit activation function for deep residual networks in short-term load forecasting

  2026cites this paper