GPLight: Grouped Multi-agent Reinforcement Learning for Large-scale Traffic Signal Control

Multi-agent reinforcement learning (MARL) method is enjoying popularity and prosperity in coordinating traffic lights (CTL), by treating each intersection as an agent. However, existing MARL approaches either treat each agent absolutely homogeneous, i.e., same network and parameter for each agent, or treat each agent completely heterogeneous, i.e., different networks and parameters for each agent. This leads to a difficult balance between accuracy and complexity, especially in large-scale CTL. To address this challenge, we propose a grouped MARL method named GPLight. We first mine the similarity between agent environment considering both real-time traffic flow and static fine-grained road topology. Then we propose two loss functions for maintaining a learnable and dynamical clustering, one applies mutual information estimation for better stability, the other aims to maximize the separability between groups. Finally, GPLight enforces the agents in a group share the same network and parameter. In this way, the cooperation between the same group of agents reduces the complexity, while different groups reflect the difference of the agents to ensure the accuracy. To verify the effectiveness of our method, we conducted experiments on both synthetic and real-world datasets, with up to 1,000 intersections. Compared with state-of-the-art methods, experiment results demonstrate the superiority of our proposed method, especially in large-scale CTL.