Multi-agent deep reinforcement learning based real-time planning approach for responsive customized bus routes

Customized bus can meet many passengers' personalized travel demand in a public transportation system by providing an innovative shared travel service. Customized bus offers multiple bus routes that jointly form a route network to serve its passengers. It must frequently adjust the station sequences of each bus route in response to trip cancellations and new trip bookings during its operation. Different from relevant research works in the literature, this paper proposes a multi-agent deep reinforcement learning based real-time planning approach for tackling the multiple customized bus routes planning problem. We model the problem as a multi -agent Markov decision process for the first time in literature where a separate agent is assigned to each route to plan its station sequence. We then develop a new multi-agent system. Each agent in the system is powered by an encoder-decoder neural network that consolidates the station sequence decision policy for each bus route. We employ a policy gradient-based reinforcement learning algorithm to train the network parameters of the multi-agent system so as to maximize the number of passengers served while ensuring the customized bus service quality and minimizing the operating cost of all customized bus routes. On three (six) problem instances in offline (online) scenarios, the trained multi-agent system can significantly outperform several existing algorithms in terms of the total cost, adaptiveness and computation time.