SCIENCE@home

Sensing, Probing, and Transmitting Policy for Energy Harvesting Cognitive Radio With Two-Stage After-State Reinforcement Learning

Keyu Wu,Hai Jiang,C. Tellambura

Published 2019 in IEEE Transactions on Vehicular Technology

ABSTRACT

This paper considers joint optimization of spectrum sensing, channel probing, and transmission power control for a single-channel secondary transmitter that operates with harvested energy from ambient sources. At each time slot, to maximize the expected secondary throughput, the transmitter needs to decide whether or not to perform the operations of spectrum sensing, channel probing, and transmission, according to energy status and channel fading status. First, we model this stochastic optimization problem as a two-stage continuous-state Markov decision process, with a sensing-and-probing stage and a transmit-power-control stage. We simplify this problem by a more useful after-state value function formulation. We then propose a reinforcement learning algorithm to learn the after-state value function from data samples when the statistical distributions of harvested energy and channel fading are unknown. Numerical results demonstrate learning characteristics and performance of the proposed algorithm.

PUBLICATION RECORD

CITATION MAP

EXTRACTION MAP

CLAIMS

  • No claims are published for this paper.

CONCEPTS

  • No concepts are published for this paper.

REFERENCES

Showing 1-36 of 36 references · Page 1 of 1

CITED BY

Showing 1-14 of 14 citing papers · Page 1 of 1