Energy-efficient operation by cooperative control among trains: A multi-agent reinforcement learning approach

With the ever-increasing operating mileage of urban rail transit systems, a large amount of energy is consumed for train operation, and the energy-saving for train operation has become an attractive topic in the research community in recent years. In order to minimize the net energy consumption (i.e., the difference between the traction energy and the reused regenerative energy), an energy-efficient train operation method using the cooperative control approach is proposed in this paper. Firstly, a set of mathematical models for the cooperative control are formulated with considering the transmission mechanism of regenerative energy. Then, a multi-agent reinforcement learning algorithm is designed to obtain the cooperative driving strategy for trains. In this algorithm, the global Q-function is decomposed into several local Q-functions by using value function factorization method. A proposed update method is applied to update the local Q-functions according to the transmission mechanism of regenerative energy. Finally, the case studies built upon a metro line is performed to illustrate the effectiveness of the proposed method. According to the simulation results, the net energy consumption is reduced by more than 11.1% compared to the method in which the driving strategy of each train is independently optimized.