Three-Stage Deep Reinforcement Learning for Privacy-and Safety-Aware Smart Electric Vehicle Charging Station Scheduling and Volt/VAR Control

This article proposes a three-stage privacy-and safety-aware deep reinforcement learning framework for coordinating smart electric vehicle charging stations (EVCSs) integrated with a photovoltaic system/ energy storage system (ESS) and volt/VAR control in a power distribution system. The proposed framework aims to maximize the EVCS profit and minimize the network real power loss while ensuring zero ESS state of charge (SOC) and voltage violation as well as preserving the privacy of the EVCS net load schedule data. In Stage 1 with 30-min resolution, each charging station operator (CSO) agent of the EVCS performs day-ahead profitable real power charging/discharging of the ESS without violating its SOC constraint via a safety layer during training. In Stage 2, using the $\epsilon $ -differential privacy method, the CSO agents encrypt the EVCS net load schedule data delivered from Stage 1. In Stage 3 with 5-min resolution, the distribution system operator agent conducts real-time reactive power charging/discharging of the ESSs to minimize the real power loss while removing voltage violations completely via iterative safe exploration of the agent with iteration penalties during training. The proposed framework was assessed on the IEEE 33-bus system for its privacy preserving and safety performances.

• Publication year

  2024

• Venue

  IEEE Internet of Things Journal

• Publication date

  2024-03-01

• Fields of study

  Computer Science, Engineering, Environmental Science

• Identifiers
  DOI 10.1109/JIOT.2023.3319588
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Privacy-Preserving and Communication-Efficient Energy Prediction Scheme Based on Federated Learning for Smart Grids

  2023cited by this paper
• Bi-Level Off-Policy Reinforcement Learning for Two-Timescale Volt/VAR Control in Active Distribution Networks

  2023cited by this paper
• Three-Stage Inverter-Based Peak Shaving and Volt-VAR Control in Active Distribution Networks Using Online Safe Deep Reinforcement Learning

  2022influential reference
• Data-Driven Coordinated Charging for Electric Vehicles With Continuous Charging Rates: A Deep Policy Gradient Approach

  2022cited by this paper
• A Decentralized Mechanism Based on Differential Privacy for Privacy-Preserving Computation in Smart Grid

  2022cited by this paper
• Preserving Privacy of Smart Meter Data in a Smart Grid Environment

  2022cited by this paper
• FedDetect: A Novel Privacy-Preserving Federated Learning Framework for Energy Theft Detection in Smart Grid

  2022cited by this paper
• Multistep Multiagent Reinforcement Learning for Optimal Energy Schedule Strategy of Charging Stations in Smart Grid

  2022cited by this paper
• A Two-Stage Scheme for Both Power Allocation and EV Charging Coordination in a Grid-Tied PV–Battery Charging Station

  2021cited by this paper
• CDDPG: A Deep-Reinforcement-Learning-Based Approach for Electric Vehicle Charging Control

  2021cited by this paper
• Attention Enabled Multi-Agent DRL for Decentralized Volt-VAR Control of Active Distribution System Using PV Inverters and SVCs

  2021cited by this paper
• Data-Driven Multi-Agent Deep Reinforcement Learning for Distribution System Decentralized Voltage Control With High Penetration of PVs

  2021cited by this paper
• Large-Scale Scenarios of Electric Vehicle Charging with a Data-Driven Model of Control

  2021cited by this paper
• Deep Reinforcement Learning for Continuous Electric Vehicles Charging Control With Dynamic User Behaviors

  2021cited by this paper
• Electric Vehicle Charging Management Based on Deep Reinforcement Learning

  2021cited by this paper
• Deep Reinforcement Learning Enabled Physical-Model-Free Two-Timescale Voltage Control Method for Active Distribution Systems

  2021cited by this paper
• Dynamic Pricing for EV Charging Stations: A Deep Reinforcement Learning Approach

  2021cited by this paper
• Cost and Pricing of Differential Privacy in Demand Reporting for Smart Grids

  2020cited by this paper
• Constrained EV Charging Scheduling Based on Safe Deep Reinforcement Learning

  2020cited by this paper
• Smart control of BESS in PV integrated EV charging station for reducing transformer overloading and providing battery-to-grid service

  2020cited by this paper
• Two-Level Distributed Volt/Var Control Using Aggregated PV Inverters in Distribution Networks

  2020cited by this paper
• Two-Stage Deep Reinforcement Learning for Inverter-Based Volt-VAR Control in Active Distribution Networks

  2020cited by this paper
• Cooperative Management for PV/ESS-Enabled Electric Vehicle Charging Stations: A Multiagent Deep Reinforcement Learning Approach

  2020cited by this paper
• California Plug-In Electric Vehicle Infrastructure Projections: 2017-2025 - Future Infrastructure Needs for Reaching the State's Zero Emission-Vehicle Deployment Goals

  2018cited by this paper
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor

  2018cited by this paper
• Cost-Friendly Differential Privacy for Smart Meters: Exploiting the Dual Roles of the Noise

  2017cited by this paper
• Adam: A Method for Stochastic Optimization

  2014cited by this paper
• Calibrating Noise to Sensitivity in Private Data Analysis

  2006cited by this paper
• Optimal sizing of capacitors placed on a radial distribution system

  1989cited by this paper

• Optimal operation of grid-friendly megawatt-level ultra-fast EV charging stations: A review on constraints, objectives and algorithms for grid-interactive operation

  2026cites this paper
• V2PCP: Toward Online Booking Mechanism for Private Charging Piles

  2025cites this paper
• The Role of Generative Artificial Intelligence in Internet of Electric Vehicles

  2025cites this paper
• Optimal Planning of EVCS Considering Renewable Energy Uncertainty via Improved Thermal Exchange Optimizer: A Practical Case Study in China

  2025cites this paper
• Reinforcement Learning for Electric Vehicle Charging Management: Theory and Applications

  2025cites this paper
• A deep reinforcement learning based charging and discharging scheduling strategy for electric vehicles

  2024cites this paper
• The Roles of Generative Artificial Intelligence in Internet of Electric Vehicles

  2024cites this paper
• SOC-Boundary and Battery Aging Aware Hierarchical Coordination of Multiple EV Aggregates Among Multi-stakeholders with Multi-Agent Constrained Deep Reinforcement Learning

  2024cites this paper