Laser Learning Environment: A new environment for coordination-critical multi-agent tasks
arxiv(2024)
摘要
We introduce the Laser Learning Environment (LLE), a collaborative
multi-agent reinforcement learning environment in which coordination is
central. In LLE, agents depend on each other to make progress
(interdependence), must jointly take specific sequences of actions to succeed
(perfect coordination), and accomplishing those joint actions does not yield
any intermediate reward (zero-incentive dynamics). The challenge of such
problems lies in the difficulty of escaping state space bottlenecks caused by
interdependence steps since escaping those bottlenecks is not rewarded. We test
multiple state-of-the-art value-based MARL algorithms against LLE and show that
they consistently fail at the collaborative task because of their inability to
escape state space bottlenecks, even though they successfully achieve perfect
coordination. We show that Q-learning extensions such as prioritized experience
replay and n-steps return hinder exploration in environments with
zero-incentive dynamics, and find that intrinsic curiosity with random network
distillation is not sufficient to escape those bottlenecks. We demonstrate the
need for novel methods to solve this problem and the relevance of LLE as
cooperative MARL benchmark.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要