Reaching Consensus in Cooperative Multi-Agent Reinforcement Learning with Goal Imagination
arxiv(2024)
摘要
Reaching consensus is key to multi-agent coordination. To accomplish a
cooperative task, agents need to coherently select optimal joint actions to
maximize the team reward. However, current cooperative multi-agent
reinforcement learning (MARL) methods usually do not explicitly take consensus
into consideration, which may cause miscoordination problem. In this paper, we
propose a model-based consensus mechanism to explicitly coordinate multiple
agents. The proposed Multi-agent Goal Imagination (MAGI) framework guides
agents to reach consensus with an Imagined common goal. The common goal is an
achievable state with high value, which is obtained by sampling from the
distribution of future states. We directly model this distribution with a
self-supervised generative model, thus alleviating the "curse of dimensinality"
problem induced by multi-agent multi-step policy rollout commonly used in
model-based methods. We show that such efficient consensus mechanism can guide
all agents cooperatively reaching valuable future states. Results on
Multi-agent Particle-Environments and Google Research Football environment
demonstrate the superiority of MAGI in both sample efficiency and performance.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要