Effective master-slave communication on a multiagent deep reinforcement learning system
Hierarchical Reinforcement Learning Workshop at the 31st Conference on NIPS, Long Beach, CA, USA(2017)
摘要
Many challenging practical problems require multiple agents to solve collaboratively. However, communication becomes a bottleneck when a multi-agent system (MAS) scales. This is particularly true when a MAS is deployed to autonomous learning (eg reinforcement learning), where massive interactive communication is required. We argue that the effectiveness of communication is a key factor to determine the intelligence level of a multi-agent learning system. In this regard, we propose to adapt the classical hierarchical master-slave architecture to facilitate efficient multi-agent communication during the interactive reinforcement learning (RL) process implemented on a deep neural network. The master agent aggregates messages uploaded from the slaves and generates unique message to each slave according to the aggregated information and the states of the slave. Each slave incorporates both the instructive messages from the master and its own to take actions to fulfill the goal. In this way, the joint action-state space of the agents grows only linearly instead of geometrically with the number of agents compared to the peer-to-peer architecture. In experiments, we show that with effective communication, the proposed multi-agent learning system consistently outperforms latest competing methods both in synthetics experiments and the challenging StarCraft 1 micromanagement tasks.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络