Automated Curriculum Reinforcement Learning in Path-finding Scenarios

Despite the broad applications of reinforcement learning in various fields, low training efficiency caused by the sparse reward problem remains crucial. In the context of pathfinding, consider the following question: given a limited number of expert trajectories, how can we find an efficient strategy that allows an agent to be trained in a shorter time achieving similar training performance in complex environments? In this paper, we propose a training strategy based on the results of inverse reinforcement learning, which not only performs well in the same environment but also reduces training time. The idea is to apply Gaussian-based inverse reinforcement learning to analyze expert trajectories, inferring reward functions and clustering. Then the framework of curriculum learning is introduced to shorten the training time and improve the training efficiency under complex tasks and extensive data. The experimental section explores the general rules for the optimal number of intermediate points. Ablation experiments were set up to demonstrate the scientific validity of the middle point selection of the curriculum. Additionally, we compare the time and reward function of our proposed method with baselines under different map sizes, numbers of obstacles, and distances to evaluate the learning performance in various scenarios. Based on the experimental results, our proposed approach outperforms the best baseline in most cases and achieves training performance similar to expert trajectories with less training time, solving the sparse reward problem.