Distributed Multi-agent Reinforcement Learning for Directional UAV Network Control

In this article, we propose a distributed medium access control (MAC) protocol for the unmanned aerial vehicle (UAV) network with directional antennas. It uses a distributed learning algorithm to control each node's communication parameters (such as sending rate) according to the observations of only 1-hop neighbors. Because of the asynchronous and decentralized nature of multi-agent deep distributed reinforcement learning (MADDRL), we adopt Multi Agent Deep Deterministic Policy Gradient (MADDPG) [3] to generate discrete actions for the MAC layer protocol. Under the individual and distributed observations of the environment, each node/agent can update their own states and rewards locally. Moreover, two major communication quality evaluation metrics - throughput (THR) and delay (DEL) are used to show the effectiveness and scalability of the distributed algorithm. From the results, the protocol using MADDPG shows more advantages than the general MAC schemes, including higher end-to-end THR, lower queuing DEL, and lower communication message exchange.