Causal Coordinated Concurrent Reinforcement Learning
CoRR(2024)
摘要
In this work, we propose a novel algorithmic framework for data sharing and
coordinated exploration for the purpose of learning more data-efficient and
better performing policies under a concurrent reinforcement learning (CRL)
setting. In contrast to other work which make the assumption that all agents
act under identical environments, we relax this restriction and instead
consider the formulation where each agent acts within an environment which
shares a global structure but also exhibits individual variations. Our
algorithm leverages a causal inference algorithm in the form of Additive Noise
Model - Mixture Model (ANM-MM) in extracting model parameters governing
individual differentials via independence enforcement. We propose a new data
sharing scheme based on a similarity measure of the extracted model parameters
and demonstrate superior learning speeds on a set of autoregressive, pendulum
and cart-pole swing-up tasks and finally, we show the effectiveness of diverse
action selection between common agents under a sparse reward setting. To the
best of our knowledge, this is the first work in considering non-identical
environments in CRL and one of the few works which seek to integrate causal
inference with reinforcement learning (RL).
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要