Towards Relational Multi-Agent Reinforcement Learning via Inductive Logic Programming

We present a relational multi-agent reinforcement learning algorithm in which two agents work together to achieve a goal in an environment represented by structured entities and relations. Our proposal takes a hybrid connectionist-symbolic approach, where a classical actor-critic method with an iterative weight update scheme is used to guide the derivation of an agent’s policy, which is purely expressed as first-order logic. A recent technique, differentiable inductive logic programming, is applied to integrate these two parts into a trainable system. We tailor the centralized training with decentralized execution framework to meet the symbolic-represented underlying structure. Agents are designed to communicate with one another in terms of logical predicates to alleviate the partially observable problem prevalent in the multi-agent setting. Empirical studies on the classical grid-world task demonstrate that the proposed method can learn close to optimal strategies and has better interpretability than traditional reinforcement learning approaches.