Extended Kalman Filter Based Resilient Formation Tracking Control of Multiple Unmanned Vehicles via Game-Theoretical Reinforcement Learning

In this paper, we discuss the resilient formation tracking control problem of multiple unmanned vehicles (MUV). A dynamic leader-follower distributed control structure is utilized to optimize the performance of the formation tracking. For the follower of the MUV, the leader is a cooperative unmanned vehicle, and the target of formation tracking is a non-cooperative unmanned vehicle with a nonlinear trajectory. Therefore, an extended Kalman filter (EKF) observer is designed to estimate the state of the target. Then the leader of the MUV is adjusted dynamically according to the state of the target. In order to describe the interactions between the follower and dynamic leader, a Stackelberg game model is constructed to handle the hierarchical decision problems. At the lower layer, each follower responds by observing the leader's strategy, and the potential game is used to prove a Nash equilibrium among all followers. At the upper layer, the dynamic leader makes decisions depending on the response of all followers to reaching the Stackelberg equilibrium. Moreover, the Stackelberg-Nash equilibrium of the designed game theoretical model is proven. A novel reinforcement learning-based algorithm is designed to achieve the Stackelberg-Nash equilibrium of the game. Finally, the effectiveness of the method is verified by a variety of formation tracking simulation experiments.