Design of Self-Organizing Systems Using Multi-Agent Reinforcement Learning and the Compromise Decision Support Problem Construct

In this paper, we address the following question: How can multi-robot self-organizing systems be designed so that they show the desired behavior and are able to perform tasks specified by the designers? Multi-robot self-organizing systems, e.g., swarm robots, have great potential for adapting when performing complex tasks in a changing environment. However, such systems are difficult to design due to the stochasticity of system performance and the non-linearity between the local actions/interaction and the desired global behavior. In order to address this, in this paper, we propose a framework for designing self-organizing systems using Multi-Agent Reinforcement Learning (MARL) and the compromise Decision-Support Problem (cDSP) construct. The proposed framework consists of two stages, namely, preliminary design followed by design improvement. In the preliminary design stage, MARL is used to help designers train the robots so that they show stable group behavior for performing the task. In the design improvement stage, the cDSP construct is used to explore the design space and identify satisfactory solutions considering several performance indicators. Surrogate models are used to map the relationship between local parameters and global performance indicators utilizing the data generated in the preliminary design. These surrogate models represent the goals of the cDSP. Our focus in this paper is to describe the framework. A multi-robot box-pushing problem is used as an example to test the framework's efficacy. This framework is general and can be extended to design other multi-robot self-organizing systems.