Comparative Analysis of Loss Functions in Deep Learning Models for Water Level Forecasting

The choice of a loss function is a fundamental factor in machine learning, as it determines how models learn from prediction errors. In time-series forecasting, this decision is closely intertwined with the architectural design used to capture temporal dependencies. This paper conducts a comprehensive comparison of deep learning architectures, including GRU, LSTM, Attention, Multi-head attention, and Transformer, under five representative loss functions (MSE, MAE, Huber, Log-Cosh, and Elastic). The analysis is carried out for water level forecasting tasks for the Red River in Hanoi and the Vu Quang station, covering multiple forecast horizons. The experimental results show that attention-based architectures, particularly the Transformer and Multi-head attention, achieve superior performance in long-term forecasting, with Multi-head attention reaching a similarity (Sim) of 0.793 at Hanoi and Transformer achieving 0.713 at Vu Quang, while recurrent models such as GRU and LSTM demonstrate more variable loss preferences across horizons, attaining similarity values of 0.739 and 0.748 respectively at long horizons. Furthermore, the study highlights that MSE and Elastic emerge as the most consistently effective objectives, especially for attention-based models. Overall, the findings emphasize the importance of jointly considering architectural design and loss function selection when developing robust forecasting systems.

• Publication year

  2025

• Venue

  2025 International Conference on Applied Artificial Intelligence, Data Engineering and Sciences (ICAIDES)

• Publication date

  2025-12-11

• Fields of study

  Not labeled

• Identifiers
  DOI 10.1109/ICAIDES67265.2025.11404066
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Evaluation of the support vector regression (SVR) and the random forest (RF) models accuracy for streamflow prediction under a data-scarce basin in Morocco

  2024cited by this paper
• Exploring machine learning algorithms for accurate water level forecasting in Muda river, Malaysia

  2023cited by this paper
• Statistical Properties of the log-cosh Loss Function Used in Machine Learning

  2022cited by this paper
• A review of machine learning concepts and methods for addressing challenges in probabilistic hydrological post-processing and forecasting

  2022cited by this paper
• A Comprehensive Review of Deep Learning Applications in Hydrology and Water Resources

  2020cited by this paper
• Combining statistical machine learning models with ARIMA for water level forecasting: The case of the Red river

  2020cited by this paper
• Rainfall–runoff modelling using Long Short-Term Memory (LSTM) networks

  2018cited by this paper
• Attention is All you Need

  2017cited by this paper
• Deep Learning

  2016cited by this paper
• Fast R-CNN

  2015cited by this paper
• Adam: A Method for Stochastic Optimization

  2014cited by this paper
• Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

  2014cited by this paper
• Neural Machine Translation by Jointly Learning to Align and Translate

  2014cited by this paper
• Addendum: Regularization and variable selection via the elastic net

  2005cited by this paper
• Long Short-Term Memory

  1997cited by this paper
• On robust estimation of the location parameter

  1980cited by this paper

• No citing papers are available for this paper.