Comprehensive Ocean Information Enabled AUV Path Planning via Reinforcement Learning

The path planning of the Autonomous Underwater Vehicle (AUV) has shown great potential in various Internet-of-Underwater-Things (IoUT) applications. Although considerable efforts had been made, prior studies are confronted with some limitations. For one thing, existing work only uses the ocean current simulation model without introducing real ocean information, having not been supported by real data. For another, traditional path planning algorithms have strong environment dependence and lack flexibility: once the environment changes, they need to be re-modelled and re-planned. To overcome these challenges, this paper proposes COID, an AUV path planning scheme exploiting comprehensive ocean information and reinforcement learning, which consists of three steps. First, we introduce the comprehensive real ocean data including weather, temperature, thermohaline, current, etc., and apply them into the regional ocean modeling system to generated reliable ocean current. Next, through well-designed state transition function and reward function, we build a 3D grid model of ocean environment for reinforcement learning. Furthermore, based on the framework of Double Dueling Deep Q Network (D3QN), COID integrates local ocean current and position features to provide state input and uses priority sampling to accelerate network convergence. The performance of COID has been evaluated and proved by numerical results, which demonstrate efficient path planning and high flexibility for expansion into different ocean environments.