Cooperative Multi-Agent Reinforcement Learning for Low-Level Wireless Communication

Traditional radio systems are strictly co-designed on the lower levels of the OSI stack for compatibility and efficiency. Although this has enabled the success of radio communications, it has also introduced lengthy standardization processes and imposed static allocation of the radio spectrum. Various initiatives have been undertaken by the research community to tackle the problem of artificial spectrum scarcity by both making frequency allocation more dynamic and building flexible radios to replace the static ones. There is reason to believe that just as computer vision and control have been overhauled by the introduction of machine learning, wireless communication can also be improved by utilizing similar techniques to increase the flexibility of wireless networks. In this work, we pose the problem of discovering low-level wireless communication schemes ex-nihilo between two agents in a fully decentralized fashion as a reinforcement learning problem. Our proposed approach uses policy gradients to learn an optimal bi-directional communication scheme and shows surprisingly sophisticated and intelligent learning behavior. We present the results of extensive experiments and an analysis of the fidelity of our approach.

• Publication year

  2018

• Venue

  arXiv.org

• Publication date

  2018-01-14

• Fields of study

  Computer Science, Engineering

• Identifiers
  arXiv 1801.04541
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Emergence of Grounded Compositional Language in Multi-Agent Populations

  2017cited by this paper
• Adaptive Switching Circuits

  2016cited by this paper
• Learning to Protect Communications with Adversarial Neural Cryptography

  2016cited by this paper
• Mastering the game of Go with deep neural networks and tree search

  2016cited by this paper
• Optimizing Expectations: From Deep Reinforcement Learning to Stochastic Computation Graphs

  2016cited by this paper
• Learning to Communicate with Deep Multi-Agent Reinforcement Learning

  2016cited by this paper
• Continuous control with deep reinforcement learning

  2015cited by this paper
• Rethinking the Inception Architecture for Computer Vision

  2015cited by this paper
• Adam: A Method for Stochastic Optimization

  2014cited by this paper
• Representation Learning: A Review and New Perspectives

  2012cited by this paper
• A Review on Spectrum Sensing for Cognitive Radio: Challenges and Solutions

  2010cited by this paper
• Dynamic Programming and Optimal Control Problem Set: Deterministic Systems and the Shortest Path Problem

  2008cited by this paper
• A tutorial on spectral clustering

  2007cited by this paper
• k-means++: the advantages of careful seeding

  2007cited by this paper
• Autonomous Helicopter Flight via Reinforcement Learning

  2003cited by this paper
• Finding the Number of Clusters in a Dataset

  2003cited by this paper
• Applications of neural networks to digital communications - a survey

  2000influential reference
• Cognitive radio: making software radios more personal

  1999cited by this paper
• Reinforcement Learning: An Introduction

  1998cited by this paper
• Least squares quantization in PCM

  1982cited by this paper

• A Review on Deep Learning Autoencoder in the Design of Next-Generation Communication Systems

  2024cites this paper
• Channel-Agnostic Training of Transmitter and Receiver for Wireless Communications

  2023influential citation
• Implementation of DNN-Based Physical-Layer Network Coding

  2023cites this paper
• Certified Policy Smoothing for Cooperative Multi-Agent Reinforcement Learning

  2022cites this paper
• Hybrid Beamforming for mmWave MU-MISO Systems Exploiting Multi-Agent Deep Reinforcement Learning

  2021cites this paper
• Adversarial Jamming for a More Effective Constellation Attack

  2021cites this paper
• Distributed Learning in Wireless Networks: Recent Progress and Future Challenges

  2021cites this paper
• Reinforcement Learning Meets Wireless Networks: A Layering Perspective

  2021cites this paper
• On the Robustness of Cooperative Multi-Agent Reinforcement Learning

  2020cites this paper
• Distributed Artificial Intelligence Solution for D2D Communication in 5G Networks

  2020cites this paper
• Challenges and Countermeasures for Adversarial Attacks on Deep Reinforcement Learning

  2020cites this paper
• Learning paradigms for communication and computing technologies in IoT systems

  2020cites this paper
• Data-Driven Solutions for Digital Communications

  2020influential citation
• Dimmer: Self-Adaptive Network-Wide Flooding with Reinforcement Learning

  2020cites this paper
• Learning Physical-Layer Communication With Quantized Feedback

  2019cites this paper
• Learning to Communicate in a Noisy Environment

  2019influential citation
• Learning to Communicate with Limited Co-design

  2019influential citation
• Backscatter Communications: Inception of the Battery-Free Era—A Comprehensive Survey

  2019cites this paper
• Effect of Model Dissimilarity on Learning to Communicate in a Wireless Setting with Limited Information

  2019influential citation
• Blind Interactive Learning of Modulation Schemes: Multi-Agent Cooperation Without Co-Design

  2019influential citation
• Model-Based: End-to-End Molecular Communication System Through Deep Reinforcement Learning Auto Encoder

  2019cites this paper
• Communication-based cooperative tasks: how the language expressiveness affects reinforcement learning

  2019cites this paper
• Application of Deep Learning Techniques to the Diagnosis of Medical Images

  2018cites this paper
• Model-Free Training of End-to-End Communication Systems

  2018cites this paper
• Deep Reinforcement Learning Autoencoder with Noisy Feedback

  2018cites this paper
• End-to-End Learning of Communications Systems Without a Channel Model

  2018cites this paper
• Real-time Communication Systems For Automation Over Wireless: Enabling Future Interactive Tech

  2018cites this paper
• Toward Anti-Jamming Constellations via Adversarial Reinforcement Learning

  year unknowncites this paper