A hybrid statistical and machine learning based forecasting framework for the energy sector

Energy prices are sensitive and volatile due to supply-demand imbalances, policy changes, new environmental targets, and technological breakthroughs. Energy security, sustainability, and affordability are key to policy design and societal advancement, making accurate energy price forecasting essential. Despite the existence of various methods for forecasting energy prices, no single method consistently outperforms the others. In this respect, the Energy Price Index (EPIC) framework is presented with extended forecasting capabilities for 56 different energy products using 33 unique time series. Statistical and machine learning forecasting methods of different natures have been incorporated into the framework, enabling the forecasting of energy prices up to 14 months ahead. Each energy product's historical prices are analyzed for seasonality, trends, and outliers. The results reveal the superiority of deep neural networks over statistical methods, while at the same time highlighting the significant importance of proper tuning of the hyperparameters of a neural network.

• Publication year

  2024

• Venue

  Computers and Chemical Engineering

• Publication date

  2024-05-01

• Fields of study

  Computer Science, Economics, Engineering, Environmental Science, Medicine

• Identifiers
  DOI 10.1016/j.compchemeng.2024.108740PMID 40247844PMCID PMC12001868
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Machine learning in process systems engineering: Challenges and opportunities

  2024cited by this paper
• Impacts of Battery Energy Storage Technologies and Renewable Integration on the Energy Transition in the New York State

  2023cited by this paper
• A novel quantitative forecasting framework in energy with applications in designing energy-intelligent tax policies

  2022cited by this paper
• A review of analytical and optimization methodologies for transitions in multi-scale energy systems

  2022cited by this paper
• Data-driven prediction and evaluation on future impact of energy transition policies in smart regions

  2022cited by this paper
• Post-script—Retail forecasting: Research and practice

  2021cited by this paper
• A framework to predict the price of energy for the end-users with applications to monetary and energy policies

  2021cited by this paper
• Short-term offshore wind speed forecast by seasonal ARIMA - A comparison against GRU and LSTM

  2021cited by this paper
• Study and analysis of SARIMA and LSTM in forecasting time series data

  2021cited by this paper
• Size Matters: Estimation Sample Length and Electricity Price Forecasting Accuracy

  2020cited by this paper
• Time-series forecasting with deep learning: a survey

  2020cited by this paper
• A hybrid method of exponential smoothing and recurrent neural networks for time series forecasting

  2020cited by this paper
• Leveraging Turbine-Level Data for Improved Probabilistic Wind Power Forecasting

  2020cited by this paper
• A novel fuzzy-based ensemble model for load forecasting using hybrid deep neural networks

  2020cited by this paper
• Forecasting Industrial Aging Processes with Machine Learning Methods

  2020cited by this paper
• LSTM based long-term energy consumption prediction with periodicity

  2020cited by this paper
• EMD–PSO–ANFIS‐based hybrid approach for short‐term load forecasting in microgrids

  2020cited by this paper
• Advanced Methods for Photovoltaic Output Power Forecasting: A Review

  2020cited by this paper
• Stock Prediction Based on LSTM under Different Stability

  2019cited by this paper
• Online adaptive lasso estimation in vector autoregressive models for high dimensional wind power forecasting

  2019cited by this paper
• Short-Term Load Forecasting in Smart Grids: An Intelligent Modular Approach

  2019cited by this paper
• Financial time series forecasting model based on CEEMDAN and LSTM

  2019cited by this paper
• Trend analysis of climate time series: A review of methods

  2019cited by this paper
• Long-term forecast of energy commodities price using machine learning

  2019cited by this paper
• Day-Ahead Photovoltaic Forecasting: A Comparison of the Most Effective Techniques

  2019cited by this paper
• On the importance of the long-term seasonal component in day-ahead electricity price forecasting with NARX neural networks

  2019cited by this paper
• Retail forecasting: Research and practice

  2019cited by this paper
• Forecasting of process disturbances using k-nearest neighbours, with an application in process control

  2019cited by this paper
• Predicting residential energy consumption using CNN-LSTM neural networks

  2019cited by this paper
• Data on forecasting energy prices using machine learning

  2019cited by this paper
• X-Model: Further Development and Possible Modifications

  2019cited by this paper
• A novel variable selection algorithm for multi-layer perceptron with elastic net

  2019cited by this paper
• Hybrid structures in time series modeling and forecasting: A review

  2019cited by this paper
• Forecasting crude oil price: Does exist an optimal econometric model?

  2018cited by this paper
• Short term electricity load forecasting using a hybrid model

  2018cited by this paper
• Comparison of Training Approaches for Photovoltaic Forecasts by Means of Machine Learning

  2018cited by this paper
• Electricity Price Forecasting Using Recurrent Neural Networks

  2018cited by this paper
• Interval decomposition ensemble approach for crude oil price forecasting

  2018cited by this paper
• Statistical and Machine Learning forecasting methods: Concerns and ways forward

  2018cited by this paper
• A Comparison of ARIMA and LSTM in Forecasting Time Series

  2018cited by this paper
• The Financial Effect of the Electricity Price Forecasts’ Inaccuracy on a Hydro-Based Generation Company

  2018cited by this paper
• Day-ahead electricity price forecasting with high-dimensional structures: Univariate vs. multivariate modeling frameworks

  2018cited by this paper
• Oil price forecasting using a hybrid model

  2018cited by this paper
• Forecasting Crude Oil Prices: a Deep Learning based Model

  2017cited by this paper
• Deep learning with long short-term memory networks for financial market predictions

  2017cited by this paper
• Probabilistic mid- and long-term electricity price forecasting

  2017cited by this paper
• Towards Improved Understanding of the Applicability of Uncertainty Forecasts in the Electric Power Industry

  2017cited by this paper
• Physical and hybrid methods comparison for the day ahead PV output power forecast

  2017cited by this paper
• Crude oil price forecasting using XGBoost

  2017cited by this paper
• Econometric Analysis of 15-Minute Intraday Electricity Prices

  2017cited by this paper
• Forecasting energy market indices with recurrent neural networks: Case study of crude oil price fluctuations

  2016cited by this paper
• Deep Learning

  2016cited by this paper
• Interrupted time series regression for the evaluation of public health interventions: a tutorial

  2016cited by this paper
• Recent advances in electricity price forecasting: A review of probabilistic forecasting

  2016cited by this paper
• Bagging exponential smoothing methods using STL decomposition and Box–Cox transformation

  2016cited by this paper
• Simple versus complex forecasting: The evidence

  2015cited by this paper
• Future opportunities and challenges in the design of new energy conversion systems

  2015cited by this paper
• Electricity price forecasting using sale and purchase curves: The X-Model

  2015cited by this paper
• Evolutionary Neural Network model for West Texas Intermediate crude oil price prediction

  2015cited by this paper
• Forecasting the Real Price of Oil in a Changing World: A Forecast Combination Approach

  2015cited by this paper
• LSTM: A Search Space Odyssey

  2015cited by this paper
• A Physical Hybrid Artificial Neural Network for Short Term Forecasting of PV Plant Power Output

  2015cited by this paper
• Electricity price forecasting: A review of the state-of-the-art with a look into the future

  2014cited by this paper
• Predicting oil price movements: A dynamic Artificial Neural Network approach

  2014cited by this paper
• Neural network ensemble operators for time series forecasting

  2014cited by this paper
• Forecasting Energy Commodity Prices Using Neural Networks

  2012cited by this paper
• Studying and modelling dynamic biological processes using time-series gene expression data

  2012cited by this paper
• A multiple adaptive wavelet recurrent neural network model to analyze crude oil prices

  2012cited by this paper
• Optimal production planning under time-sensitive electricity prices for continuous power-intensive processes

  2012cited by this paper
• Forecasting Time Series With Complex Seasonal Patterns Using Exponential Smoothing

  2011cited by this paper
• Electricity price forecasting using artificial neural networks

  2011cited by this paper
• Day-ahead electricity price forecasting using wavelet transform combined with ARIMA and GARCH models

  2010cited by this paper
• An Empirical Comparison of Machine Learning Models for Time Series Forecasting

  2010cited by this paper
• Feature selection for time series prediction - A combined filter and wrapper approach for neural networks

  2010cited by this paper
• International evidence on crude oil price dynamics: Applications of ARIMA-GARCH models

  2010cited by this paper
• Time Series Prediction Using Support Vector Machines: A Survey

  2009cited by this paper
• Time series forecasting with neural networks: a comparative study using the air line data

  2008cited by this paper
• Automatic Time Series Forecasting: The forecast Package for R

  2008cited by this paper
• Periodic Seasonal Reg-ARFIMA–GARCH Models for Daily Electricity Spot Prices

  2007cited by this paper
• Extreme learning machine: Theory and applications

  2006cited by this paper
• A New Method for Crude Oil Price Forecasting Based on Support Vector Machines

  2006cited by this paper
• A comparative predictive analysis of neural networks (NNs), nonlinear regression and classification and regression tree (CART) models

  2005cited by this paper
• Forecasting seasonals and trends by exponentially weighted moving averages

  2004cited by this paper
• Gaussian process for nonstationary time series prediction

  2004cited by this paper
• FORECASTING WITH ARTIFICIAL NEURAL NETWORKS: THE STATE OF THE ART

  1998cited by this paper
• Physiological time-series analysis: what does regularity quantify?

  1994cited by this paper
• Time‐series analysis supported by power transformations

  1993cited by this paper
• Generalized autoregressive conditional heteroskedasticity

  1986cited by this paper
• Exponential smoothing: The state of the art

  1985cited by this paper
• Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation

  1982cited by this paper
• An Analysis of Transformations

  1964cited by this paper
• Forecasting Sales by Exponentially Weighted Moving Averages

  1960cited by this paper
• On a Method of Investigating Periodicities in Disturbed Series, with Special Reference to Wolfer's Sunspot Numbers

  year unknowncited by this paper

• Global renewable energy forecasting using hybrid ML/DL models: Economic and geospatial insights

  2026cites this paper
• A Comprehensive Approach to Defining the Cost of Inventory Management: A Case Study on Small Batch Cargo Delivery

  2026cites this paper
• Data, models, algorithms, AI and the role of PSE - the generation next

  2026cites this paper
• Planning strategies in the energy sector: Integrating bayesian neural networks and uncertainty quantification in scenario analysis & optimization

  2025cites this paper
• A Perspective on Artificial Intelligence for Process Manufacturing

  2025cites this paper
• AI-based predictive analytics for enhancing data-driven supply chain optimization

  2025cites this paper
• Integrated Intelligent Wood Management for Forecasting and Route Optimization

  2025influential citation
• Evaluation of the Optimal Features and Machine Learning Algorithms for Energy Yield Forecasting of a Rural Rooftop PV Installation

  2024cites this paper
• Enhanced Convolution Neural Network with Optimized Pooling and Hyperparameter Tuning for Network Intrusion Detection

  2024cites this paper
• Framework for multivariate carbon price forecasting: A novel hybrid model.

  2024cites this paper
• Models, modeling and model-based systems in the era of computers, machine learning and AI

  2024cites this paper