Multi-Agent Deep Reinforcement Learning for Charging Coordination of Electric Vehicles

Virtual power plant (VPP) is a power system control center that mediates among the energy market, power grid, distributed energy resources, power storage units, controllable power loads, and electric vehicles (EV s), As energy consumers and wireless communication users, an EV is one of the key components of VPP systems that pose stringent requirements on both charging system management and communication infrastructure. On one hand, the optimal power flow (OPF) needs to minimize power system loss and voltage variance. On the other hand, communication quality of service (QoS) needs to be accounted for, in terms of data rate and latency in VPP systems. These lead long-term spatial EV charging coordination (i.e., charging station (CS) selection) a non-convex optimization problem, which is yet an open issue. In this regard, by embedding SG millimeter-wave (mm W) access point (AP) into CS, an AP-enabled CS architecture is first devised in this paper to support VPP control information flow and EV communication applications, such as autonomous driving. Based on the AP-enabled CS, an AP-enabled-CS-assisted multi-agent deep reinforcement learning algorithm (ABC-MADRL) is proposed to provide an intelligent long-term EV charging coordination solution. Experimental results illustrate that ABC-MADRL dramatically reduces the power loss and voltage variance by 24.3%, compared to the uncoordinated charging.