A Novel Distributed Reinforcement Learning Training System for Network Congestion Control

Network communication is the foundation of various industries and the Internet. As the core of network communication, congestion control (CC) algorithms directly impact network quality. Traditional CC algorithms, such as CUBIC and BBR, are designed by experts. With the advancement of deep reinforcement learning (RL) technology, many researchers have designed reinforcement learning algorithms to achieve self-learning models. However, existing RL models are limited to generating training samples on a single machine, which restricts the training efficiency of the. Additionally, during the model training process, the lack of online analysis tools makes it difficult to analyze the training failures of the model. In this paper, we propose an RL-distributed training system for CC, which utilizes multiple machines to generate training samples. Furthermore, we have designed an online analysis kit to observe the model's control performance in real-time. In experiments, we successfully replicated Indigo using the distributed training system. The experimental results show the effectiveness of the system and the online analysis kit.