Distributed Reinforcement Learning for Networked Dynamical Systems

We propose a scalable algorithm for learning distributed optimal controllers for networked dynamical systems. Assuming that the network is comprised of nearly homogeneous subsystems, each sub-controller is trained by the local state and input information from its corresponding subsystem and filtered information from its neighbors. Thus, the costs of both learning and control become independent of the number of subsystems. We show the optimality and convergence of the algorithm for the case when the individual subsystems are identical, based on an algebraic property of such networks. Thereafter, we show the robustness of the algorithm when applied to general heterogeneous networks. The effectiveness of the design is investigated through numerical simulations.