Communicating Unexpectedness for Out-of-Distribution Multi-Agent Reinforcement Learning

Applying multi-agent reinforcement learning (MARL) to real-world scenarios is challenging because agents often need to adapt quickly to unexpected situations, including those rarely or never encountered in training. Recent methods for out-of-distribution generalization are unsuitable for applications on out-of-distribution tasks with limited communication, because they are typically restricted to centralized training or some specialized instances of distribution shifts. To address this limitation, we introduce the Unexpectedness Encoding Scheme, a new decentralized MARL algorithm in which agents communicate “unexpectedness,” the surprising aspects of the environment. In addition to sending their usual reward-driven messages, each agent predicts the next observation based on past experience and then compares this prediction with the actual outcome. The discrepancy between the two is encoded as a message, enabling agents to adapt more effectively to sudden or extreme changes. Experimental results on multi-agent cooperative tasks demonstrate that our method adapts robustly to both dynamically changing training environments and previously unseen out-of-distribution scenarios.

• Publication year

  2026

• Venue

  IEEE Access

• Publication date

  Unknown publication date

• Fields of study

  Computer Science

• Identifiers
  DOI 10.1109/ACCESS.2026.3660261
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Sample Complexity of Distributionally Robust Off-Dynamics Reinforcement Learning with Online Interaction

  2025cited by this paper
• Revisiting Data Augmentation in Deep Reinforcement Learning

  2024cited by this paper
• A survey on multi-agent reinforcement learning and its application

  2024cited by this paper
• Multi-Agent Reinforcement Learning Based UAV Swarm Communications Against Jamming

  2023cited by this paper
• Toward More Human-Like AI Communication: A Review of Emergent Communication Research

  2023cited by this paper
• A survey of multi-agent deep reinforcement learning with communication

  2022cited by this paper
• Multistep Multiagent Reinforcement Learning for Optimal Energy Schedule Strategy of Charging Stations in Smart Grid

  2022cited by this paper
• Promoting Resilience in Multi-Agent Reinforcement Learning via Confusion-Based Communication

  2021cited by this paper
• Multi-target tracking for unmanned aerial vehicle swarms using deep reinforcement learning

  2021cited by this paper
• Mis-spoke or mis-lead: Achieving Robustness in Multi-Agent Communicative Reinforcement Learning

  2021cited by this paper
• A Survey of Zero-shot Generalisation in Deep Reinforcement Learning

  2021cited by this paper
• Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks

  2020influential reference
• QPLEX: Duplex Dueling Multi-Agent Q-Learning

  2020cited by this paper
• Mastering Atari, Go, chess and shogi by planning with a learned model

  2019cited by this paper
• Learning Transferable Cooperative Behavior in Multi-Agent Teams

  2019cited by this paper
• Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning

  2019cited by this paper
• Grandmaster level in StarCraft II using multi-agent reinforcement learning

  2019cited by this paper
• Dota 2 with Large Scale Deep Reinforcement Learning

  2019cited by this paper
• Value-Decomposition Networks For Cooperative Multi-Agent Learning Based On Team Reward

  2018cited by this paper
• Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning

  2018cited by this paper
• Learning dexterous in-hand manipulation

  2018cited by this paper
• Learning Attentional Communication for Multi-Agent Cooperation

  2018cited by this paper
• Proximal Policy Optimization Algorithms

  2017influential reference
• Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments

  2017cited by this paper
• A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity

  2017cited by this paper
• Counterfactual Multi-Agent Policy Gradients

  2017cited by this paper
• High-Dimensional Continuous Control Using Generalized Advantage Estimation

  2015cited by this paper
• On the Properties of Neural Machine Translation: Encoder–Decoder Approaches

  2014cited by this paper
• Adam: A Method for Stochastic Optimization

  2014cited by this paper
• Optimal and Approximate Q-value Functions for Decentralized POMDPs

  2008cited by this paper
• Markov Games as a Framework for Multi-Agent Reinforcement Learning

  1994cited by this paper

• No citing papers are available for this paper.