A Kind of Joint Routing and Resource Allocation Scheme Based on Prioritized Memories-Deep Q Network for Cognitive Radio Ad Hoc Networks

Cognitive Radio (CR) is a promising technology to overcome spectrum scarcity, which currently faces lots of unsolved problems. One of the critical challenges for setting up such systems is how to coordinate multiple protocol layers such as routing and spectrum access in a partially observable environment. In this paper, a deep reinforcement learning approach is adopted for solving above problem. Firstly, for the purpose of compressing huge action space in the cross-layer design problem, a novel concept named responsibility rating is introduced to help decide the transmission power of every Secondary User (SU). In order to deal with problem of dimension curse while reducing replay memory, the Prioritized Memories Deep Q-Network (PM-DQN) is proposed. Furthermore, PM-DQN is applied to solve the joint routing and resource allocation problem in cognitive radio ad hoc network for minimizing the transmission delay and power consumption. Simulation results illustrates that our proposed algorithm can reduce the end-to-end delay, packet loss ratio and estimation error while achieving higher energy efficiency compared with traditional algorithm.

• Publication year

  2018

• Venue

  Italian National Conference on Sensors

• Publication date

  2018-07-01

• Fields of study

  Medicine, Computer Science, Engineering

• Identifiers
  DOI 10.3390/s18072119PMID 30004424PMCID 6068577
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• A Distributed Learning Automata Scheme for Spectrum Management in Self-Organized Cognitive Radio Network

  2017cited by this paper
• An Energy-Efficient and Robust Multipath Routing Protocol for Cognitive Radio Ad Hoc Networks

  2017cited by this paper
• Clustering and Reinforcement-Learning-Based Routing for Cognitive Radio Networks

  2017cited by this paper
• Medium Access Control Protocols for Cognitive Radio Ad Hoc Networks: A Survey

  2017cited by this paper
• Variable Admittance Control Based on Fuzzy Reinforcement Learning for Minimally Invasive Surgery Manipulator

  2017cited by this paper
• Efficient Resource Allocation in Device-to-Device Communication Using Cognitive Radio Technology

  2017cited by this paper
• Distributed Heuristically Accelerated Q-Learning for Robust Cognitive Spectrum Management in LTE Cellular Systems

  2016cited by this paper
• Prioritized Experience Replay

  2015cited by this paper
• Cooperative Routing for Underlay Cognitive Radio Networks Using Mutual-Information Accumulation

  2015cited by this paper
• Human-level control through deep reinforcement learning

  2015cited by this paper
• A Survey on Distributed Channel Selection Technique Using Surf Algorithm for Information Transfer in Multi-hop Cognitive Radio Networks

  2014cited by this paper
• Joint Routing and Resource Allocation for Delay Minimization in Cognitive Radio Based Mesh Networks

  2014cited by this paper
• Reinforcement Learning for Routing in Cognitive Radio Ad Hoc Networks

  2014cited by this paper
• Multi-Agent Reinforcement Learning Based Opportunistic Routing and Channel Assignment for Mobile Cognitive Radio Ad Hoc Network

  2014cited by this paper
• A reinforcement learning-based routing scheme for cognitive radio ad hoc networks

  2014cited by this paper
• Cognitive routing for multi-hop mobile cognitive radio ad hoc networks

  2014cited by this paper
• An Introduction to Reinforcement Learning

  2013cited by this paper
• Stochastic Power Adaptation with Multiagent Reinforcement Learning for Cognitive Wireless Mesh Networks

  2013cited by this paper
• Spectrum-Aware Opportunistic Routing in Multi-Hop Cognitive Radio Networks

  2012cited by this paper
• Routing in cognitive radio networks: Challenges and solutions

  2011cited by this paper
• Cross-Layer Routing and Dynamic Spectrum Allocation in Cognitive Radio Ad Hoc Networks

  2010cited by this paper
• Evaluation of Adaptive MAC-Layer Sensing in Realistic Spectrum Occupancy Scenarios

  2010cited by this paper
• Efficient Discovery of Spectrum Opportunities with MAC-Layer Sensing in Cognitive Radio Networks

  2008cited by this paper
• A review of current routing protocols for ad hoc mobile wireless networks

  1999cited by this paper
• Generalized Prioritized Sweeping

  1997cited by this paper
• A Note on the Generation of Random Normal Deviates

  1958cited by this paper
• Dynamic Programming

  1957cited by this paper

• A Hierarchical Traffic Offloading Mechanism for End-to-End Reliability in a Multihop Multiconnection Wireless Sensor Network

  2025cites this paper
• Deep-EERA: DRL-Based Energy-Efficient Resource Allocation in UAV-Empowered Beyond 5G Networks

  2025cites this paper
• Spectral Efficiency Maximization for V2V Communication Underlaid Cellular Uplink Using Deep Neural Networks

  2023cites this paper
• Dynamic Feature Selection for Solar Irradiance Forecasting Based on Deep Reinforcement Learning

  2023cites this paper
• Review of unmanned cluster routing protocols based on deep reinforcement learning

  2023cites this paper
• An efficient resource optimization scheme for D2D communication

  2022cites this paper
• Resource Allocation and Information Exchange of Cognitive user Connectivity with Minimal Interference using Simulation Analysis

  2022cites this paper
• DHL: Deep reinforcement learning-based approach for emergency supply distribution in humanitarian logistics

  2022cites this paper
• DNN-Based Dynamic Transmit Power Control for V2V Communication Underlaid Cellular Uplink

  2022cites this paper
• Data Collection and Analysis of Track and Field Athletes' Behavior Based on Edge Computing and Reinforcement Learning

  2021cites this paper
• Review on Intelligent Optimization Technology for Survivability of Power Grid Backbone Communication Networks

  2021cites this paper
• Throughput Maximization by Deep Reinforcement Learning With Energy Cooperation for Renewable Ultradense IoT Networks

  2020cites this paper
• The Application of Fog Computing and Internet of Things Technology in Music Resource Management Model

  2020cites this paper
• Survivability-aware routing restoration mechanism for smart grid communication network in large-scale failures

  2020cites this paper
• Power Control and Routing Selection for Throughput Maximization in Energy Harvesting Cognitive Radio Networks

  2020cites this paper
• Fast Learning Cognitive Radios in Underlay Dynamic Spectrum Access: Integration of Transfer Learning into Deep Reinforcement Learning

  2020cites this paper
• Queue-aware QoE-based Rate and Power Adaptation Over Underlay Cognitive Radio Networks: Cross-Layer Design

  2020cites this paper
• A Cross-Layer Routing Protocol Based on Quasi-Cooperative Multi-Agent Learning for Multi-Hop Cognitive Radio Networks

  2019influential citation
• A Hierarchical Routing Protocol based on Energy Consumption Weight Clustering Scheme for Cognitive Radio Networks

  2019cites this paper
• An Anti-Jamming Communication Scheme Based on Global Performance for Multi-Hop Cognitive Radio Networks

  2019cites this paper
• Energy Aware Resource Allocation in Multi-Hop Multimedia Routing via the Smart Edge Device

  2019cites this paper
• An Energy-Efficient Cross-Layer Routing Protocol for Cognitive Radio Networks Using Apprenticeship Deep Reinforcement Learning

  2019cites this paper
• An apprenticeship learning scheme based on expert demonstrations for cross-layer routing design in cognitive radio networks

  2019cites this paper
• A route stability-based multipath QoS routing protocol in cognitive radio ad hoc networks

  2019cites this paper
• Deep Reinforcement Learning Optimal Transmission Policy for Communication Systems With Energy Harvesting and Adaptive MQAM

  2019cites this paper