Energy-Efficient Sleep Mode Optimization in 5G mmWave Networks via Multi-Agent Deep Reinforcement Learning

Dynamic sleep mode optimization (SMO) in millimeter-wave (mmWave) networks is essential for maximizing energy efficiency (EE) under stringent quality-of-service (QoS) constraints. However, existing optimization and reinforcement learning (RL)-based approaches rely on aggregated, static base station (BS) traffic models that fail to capture non-stationary traffic dynamics and suffer from prohibitively large state–action spaces, limiting their real-world deployment. To address these challenges, this paper proposes a Multi-Agent Deep Reinforcement Learning (MARL) framework employing a Double Deep Q-Network (DDQN), referred to as MARL-DDQN, for adaptive SMO in a 3D urban environment using a time-varying and community-based user equipment (UE) mobility model. Unlike conventional single-agent RL, the proposed MARL-DDQN enables scalable, distributed decision-making with minimal signaling overhead. A realistic BS power consumption model and beamforming are integrated to accurately quantify EE, while QoS is uniquely defined in terms of throughput. The proposed method adaptively learns SMO policies to maximize EE while mitigating inter-cell interference and ensuring throughput fairness. Extensive simulations demonstrate that MARL-DDQN consistently outperforms state-of-the-art SM strategies, including the All On, iterative QoS-aware load-based (IT-QoS-LB), MARL-DDPG, and MARL-PPO, achieving up to 0.60 Mbit/Joule EE, 8.5 Mbps 10th-percentile throughput, and satisfying QoS constraints 95% of the time under dynamic network scenarios.

• Publication year

  2025

• Venue

  IEEE Transactions on Green Communications and Networking

• Publication date

  2025-11-27

• Fields of study

  Computer Science, Engineering

• Identifiers
  DOI 10.1109/TGCN.2025.3638812arXiv 2511.22105
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Energy-Efficient Sleep Mode Optimization of 5G mmWave Networks Using Deep Contextual MAB

  2024influential reference
• Optimization of mobile‐relay‐aided base station sleep mode in wireless dense small cell network

  2023cited by this paper
• Using Reinforcement Learning to Reduce Energy Consumption of Ultra-Dense Networks With 5G Use Cases Requirements

  2023cited by this paper
• Deep Reinforcement Learning With Spatio-Temporal Traffic Forecasting for Data-Driven Base Station Sleep Control

  2021cited by this paper
• Joint Traffic Prediction and Base Station Sleeping for Energy Saving in Cellular Networks

  2021cited by this paper
• 3GPP TR 38.901 Channel Model

  2021influential reference
• Increasing energy efficiency of Massive-MIMO network via base stations switching using reinforcement learning and radio environment maps

  2021cited by this paper
• Reinforcement Learning for Traffic-Adaptive Sleep Mode Management in 5G Networks

  2020cited by this paper
• Base Station and Passive Reflectors Placement for Urban mmWave Networks

  2020cited by this paper
• Reinforcement Learning for Delay-Constrained Energy-Aware Small Cells with Multi-Sleeping Control

  2020cited by this paper
• Location-Aware Sleep Strategy for Energy-Delay Tradeoffs in 5G with Reinforcement Learning

  2019cited by this paper
• A Distributed Q-Learning Approach for Adaptive Sleep Modes in 5G Networks

  2019cited by this paper
• Load-Based On/Off Scheduling for Energy-Efficient Delay-Tolerant 5G Networks

  2018influential reference
• Dynamic Base Station Sleep Control and RF Chain Activation for Energy-Efficient Millimeter-Wave Cellular Systems

  2018cited by this paper
• DRAG: Deep Reinforcement Learning Based Base Station Activation in Heterogeneous Networks

  2018cited by this paper
• Energy-Efficient and Load-Proportional eNodeB for 5G User-Centric Networks: A Multilevel Sleep Strategy Mechanism

  2018cited by this paper
• Analytical approach to base station sleep mode power consumption and sleep depth

  2017cited by this paper
• Optimization of Base-Station Density for High Energy-Efficient Cellular Networks With Sleeping Strategies

  2016cited by this paper
• Small Cell Base Station Sleep Strategies for Energy Efficiency

  2016cited by this paper
• Analysis of Handover Failures in Heterogeneous Networks With Fading

  2015cited by this paper
• Energy-optimal base station density in cellular access networks with sleep modes

  2015cited by this paper
• Ultra-dense networks in millimeter-wave frequencies

  2015cited by this paper
• Multi-armed bandits with application to 5G small cells

  2015cited by this paper
• Limited Feedback Hybrid Precoding for Multi-User Millimeter Wave Systems

  2014cited by this paper
• Image georectification and feature tracking toolbox: ImGRAFT

  2014cited by this paper
• GreenBSN: Enabling Energy-Proportional Cellular Base Station Networks

  2014cited by this paper
• Optimal wake-up mechanism for single base station with sleep mode

  2013cited by this paper
• Toward dynamic energy-efficient operation of cellular network infrastructure

  2011cited by this paper
• Minimizing Energy Consumption via Sleep Mode in Green Base Station

  2010cited by this paper
• OpenStreetMap: User-Generated Street Maps

  2008cited by this paper
• Modeling Time-Variant User Mobility in Wireless Mobile Networks

  2007cited by this paper
• Reinforcement Learning: An Introduction

  1998cited by this paper

• No citing papers are available for this paper.