Pursuit-Evasion Game of Unmanded Surface Vehicles Based on Deep Reinforcement Learning

Making decisions during the pursuit-evasion game of unmanned surface vehicles (USVs) in a restricted environment with obstacles is a challenging problem. Specifically, in the pursuit game, the pursuer needs to consider how to approach the evader quickly and how to surround the evader and safely avoid obstacles in an environment containing obstacles. This paper proposes a distributed algorithm based on deep reinforcement learning to help USV solve the pursuit problem in a restricted environment. The proposed algorithm can deal with the game problem of multiple USVs on the ocean’s surface. In particular, composite reward function with guiding characteristics are designed based on the artificial potential field (APF) method for pursuit, encirclement, and obstacle avoidance, which can help the USV improve pursuit performance. Then, curriculum learning is used to help the USV improve learning efficiency in the early stage. The simulation results show that the algorithm is effective in different initial conditions and performs well.