Intelligent Trajectory Design and Charging Scheduling in Wireless Rechargeable Sensor Networks with Obstacles

Wireless rechargeable sensor networks (WRSNs) are promising in maintaining sustainable large-area monitoring tasks. Mobile chargers (MCs) are commonly used in WRSNs to replenish energy to nodes due to its flexibility and easy maintenance. Most existing works on WRSNs focus on designing offline or model-based online charging methods, which need the exact system information to conduct the optimization. However, in practical WRSNs, the exact system information such as the nodes' locations and energy consumption rates may not be easily accessible to the optimizer due to their unpredictability and high dynamics. Thus, in this work, we jointly optimize the MC's trajectory design and charging scheduling in a general and practical WRSN with inaccessibility to the exact system information, such that the charging utility of the MC is maximized. To address this problem, we introduce the model-free reinforcement learning (RL) technique, which enables the MC to learn to jointly optimize its moving trajectory and charging scheduling by interacting with the environment and tracking feedback signals from nodes and obstacles in real time. Specifically, we develop a soft actor-critic based mobile security policy intervened algorithm (SAC-MSPI) based on a novel safe RL framework, which maximizes the MC's charging utility while maintaining the safe movement (not hitting obstacles) for the MC during the entire charging period. Extensive evaluation results show that the proposed SAC-MSPI algorithm outperforms existing main RL solutions and traditional algorithms with respect to the charging utility maximization as well as the collision avoidance.