Framework of Active Obstacle Avoidance for Autonomous Vehicle Based on Hybrid Soft Actor-Critic Algorithm

In this paper, a framework of active obstacle avoidance for autonomous vehicles based on the hybrid soft actor-critic (SAC) algorithm is proposed. In the stage of local path planning, a comprehensive cost function considering collision risks, deviation from the global route, road lines crossing, and driving comfortability is developed to provide a local optimal path avoiding both static and dynamic obstacles considering multiple predicting timesteps. Then, a path tracking controller on the foundation of a hybrid SAC algorithm is designed to mitigate the problem of high sample complexity caused by random initialization of parameters in conventional reinforcement learning approaches. Model predictive control (MPC) plays a guiding role by applying its control action to combine with the action of SAC online to obtain a more effective state and reward information for training. The mechanism of the combination of MPC with SAC to balance the exploration and reliability is explained in detail. In order to improve the convergence rate and learning efficiency, a dual actor network structure for two different control actions is adopted. With considerations of various relevant factors influencing the control effect, the reward for the hybrid SAC algorithm is designed carefully. Finally, the results of simulation experiments illustrate that the proposed approach performs effectively with the assurance of safety and driving comfortability. In summary, the hybrid SAC algorithm with dual actor networks performs better than other algorithms for comparison in all test scenarios in this paper.