Multi-Agent Learning Based Packet Routing in Multi-Hop UAV Relay Network

In this paper, we investigate the packet routing problem in a multi-hop unmanned aerial vehicles (UAV) relay network, where multiple UAVs serve as relays between a base station (BS) and remote ground users (GUs) for enhancing network throughput. The challenges are that the dynamic network topology due to UAV mobility leads to volatile wireless connection. Moreover, there exists strong interference among UAVs due to line-of-sight communication links. Towards this end, we propose a novel multi-agent deep reinforcement learning based algorithm, named as MAQMIX for: 1) designing proper UAVs' trajectories to provide reliable network connection between the BS and remote GUs; 2) properly allocating frequency resource among UAVs to alleviate interference; 3) choosing a proper next-hop UAV for each data packet. Specifically, two training mechanisms are incorporated in the MAQMIX, i.e., intra UAV and inter UAV training mechanisms, which can tackle large action space issue and coordinate the training among UAVs, respectively. Simulation results show that the MAQMIX can significantly outperform baseline schemes in terms of transmission time and network throughput.