End-to-End Stochastic Predict-Then-Optimize for Cost-Efficient Water-Energy Resource Scheduling

Compared to other energy-intensive industries, urban water supply systems are unique due to the strong temporal coupling with the power grid, which intensifies grid stress during peak hours. This paper addresses the water-energy nexus by developing a water-energy resource scheduling model for whole-process water supply that incorporates electricity cost prediction uncertainties. Meanwhile, to minimize the scheduling cost in an end-to-end approach, we propose the Stochastic Predict-then-Optimize (SPTO), which aligns prediction distribution with the optimality gap by integrating probabilistic forecasting with stochastic optimization. Unlike traditional two-stage approaches, SPTO establishes a closed-loop feedback between forecasting and optimization, reducing cumulative errors and improving real-time adaptability. To enable end-to-end training, we derive the SPTO+, a differentiable convex upper-bound surrogate function based on Lagrangian duality, which provides surrogate gradient computation for end-to-end training. Experiments on real-world systems demonstrate that SPTO significantly reduces operational cost and decision regret, effectively coordinating water operations with dynamic grid conditions to support cost-efficient and grid-responsive scheduling.

• Publication year

  2025

• Venue

  IEEE Transactions on Smart Grid

• Publication date

  2025-11-01

• Fields of study

  Computer Science, Engineering, Environmental Science

• Identifiers
  DOI 10.1109/TSG.2025.3601238
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Competitive Incentive Mechanism for Multi-Agents in Demand Response via a Hierarchical Game Considering Joint Uncertainties

  2025cited by this paper
• Resilience Assessment of Urban Power Network Considering Multi-Stage Electrical-Water Fault Propagation

  2025cited by this paper
• Distributed Deep Reinforcement Learning for Data-Driven Water Heater Model in Smart Grid

  2025cited by this paper
• Assessing Power and Water Network Resilience When Water Pumps Provide Frequency Regulation

  2025cited by this paper
• Knowledge-based Bi-correction model for achieving effective lag-free characteristic on daily urban water demand forecasting

  2024cited by this paper
• Integrating end-to-end prediction-with-optimization for distributed hydrogen energy system scheduling*

  2024cited by this paper
• Electricity demand forecasting at distribution and household levels using explainable causal graph neural network

  2024cited by this paper
• An End-to-End Predict-Then-Optimize Clustering Method for Stochastic Assignment Problems

  2024cited by this paper
• Learning and Optimization for Price-Based Demand Response of Electric Vehicle Charging

  2024cited by this paper
• CritiCoder: An End-to-End Uncertain Regression Network for Robust Macroscopic Pressure Models in Water Distribution Systems

  2024cited by this paper
• Rethinking and Benchmarking Predict-then-Optimize Paradigm for Combinatorial Optimization Problems

  2023cited by this paper
• Modeling of Water Transfer Delay for Optimal Operation of Integrated Electrical-Water System

  2023cited by this paper
• Dynamic Modelling and Mutual Coordination of Electricity and Watershed Networks for Spatio-Temporal Operational Flexibility Enhancement Under Rainy Climates

  2023cited by this paper
• Distribution robust water-based demand side management in power transmission networks

  2023cited by this paper
• Hard-coupling water and power system models increases the complementarity of renewable energy sources

  2022cited by this paper
• Quantifying the Impacts of Modelling Assumptions on Accuracy and Computational Efficiency for Integrated Water-Energy System Simulations Under Uncertain Climate

  2022cited by this paper
• PyEPO: a PyTorch-based end-to-end predict-then-optimize library for linear and integer programming

  2022cited by this paper
• Polyhedral Relaxations for Optimal Pump Scheduling of Potable Water Distribution Networks

  2022cited by this paper
• SurCo: Learning Linear Surrogates For Combinatorial Nonlinear Optimization Problems

  2022cited by this paper
• Cyber-Physical Systems for Smart Water Networks: A Review

  2021cited by this paper
• Short-term electricity load and price forecasting by a new optimal LSTM-NN based prediction algorithm

  2021cited by this paper
• Integrated Electric Power/Water Distribution System Modeling and Control Under Extreme Mega Drought Scenarios

  2021cited by this paper
• Chance-Constrained Water Pumping to Manage Water and Power Demand Uncertainty in Distribution Networks

  2020cited by this paper
• Interior Point Solving for LP-based prediction+optimisation

  2020cited by this paper
• Optimal Coordinated Operation of Interdependent Power and Water Distribution Systems

  2020cited by this paper
• Smart Predict-and-Optimize for Hard Combinatorial Optimization Problems

  2019cited by this paper
• Modeling and Optimization of Time-of-Use Electricity Pricing Systems

  2019cited by this paper
• DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks

  2017cited by this paper
• Smart "Predict, then Optimize"

  2017influential reference
• Task-based End-to-end Model Learning in Stochastic Optimization

  2017cited by this paper
• OptNet: Differentiable Optimization as a Layer in Neural Networks

  2017cited by this paper
• Attention is All you Need

  2017cited by this paper
• Optimization of Potable Water Distribution and Wastewater Collection Networks: A Systematic Review and Future Research Directions

  2016cited by this paper

• Integrated optimal operation of power and water systems under uncertainty: An adjustable robust optimization approach.

  2026cites this paper