In complex social-ecological systems, multiple agents with diverse objectives take actions that affect the long-term dynamics of the system. Common pool resources are a subset of such systems, where property rights are typically poorly defined and dynamics are unknown a priori, creating a social dilemma reflected by the well-known ‘tragedy of the commons.’ In this paper, we investigated the efficacy of deep reinforcement learning in a multi-agent setting of a common pool resource system. We used an abstract mathematical model of the system, represented as a partially-observable general-sum Markov game. In the first set of experiments, the independent agents used a deep Q-Network with discrete action spaces to guide decision-making. However, significant shortcomings were evident. Consequently, in a second set of experiments, a Deep Deterministic Policy Gradient learning model with continuous state and action spaces guided agent learning. Simulation results show that agents performed significantly better in terms of both sustainability and economic goals when using the second deep learning model. Despite the fact that agents do not have perfect foresight nor understanding of the implications of their ‘harvesting’ efforts, deep reinforcement learning can be used effectively to ‘learn in the commons.’
Deep Multi-agent Reinforcement Learning in a Common-Pool Resource System
Published 2019 in IEEE Congress on Evolutionary Computation
ABSTRACT
PUBLICATION RECORD
- Publication year
2019
- Venue
IEEE Congress on Evolutionary Computation
- Publication date
2019-06-01
- Fields of study
Computer Science, Economics, Environmental Science
- Identifiers
- External record
- Source metadata
Semantic Scholar
CITATION MAP
EXTRACTION MAP
CLAIMS
- No claims are published for this paper.
CONCEPTS
- No concepts are published for this paper.
REFERENCES
Showing 1-37 of 37 references · Page 1 of 1
CITED BY
Showing 1-5 of 5 citing papers · Page 1 of 1