Joint optimization of trajectory, beamforming, and power allocation in UAV-enabled WPT networks using DRL combined with water-filling algorithm

In this work, we study a wireless power transfer (WPT) network supported by an unmanned aerial vehicle (UAV), in which a UAV equipped with an array of antennas supplies wireless energy to charge ground user (GU) devices. With this configuration, we aim to maximize the lowest GU energy by jointly optimizing the UAV's trajectory, beamforming pattern, and transmitting power. Because the proposed maximization of the lowest GU energy is a non-convex optimization problem that is difficult to be solved, we transform the problem into a grid world problem in discrete-time space and propose a design of a deep reinforcement learning (DRL) approach that optimizes the loss function by determining the UAV's moving direction, beamforming angle, and transmit power level. Furthermore, we combine the water-filling algorithm with DRL, which can assist DRL in determining the hovering duration. Simulation results verify that our proposed design increases the minimum energy of GUs received from the UAV compared to the successive hover-and-fly algorithm while exploring the most appropriate path with low computational complexity.