Bankruptcy-evolutionary games based solution for the multi-agent credit assignment problem

Multi-agent Credit Assignment (MCA) problem is considered as one of the critical challenges in developing Multi-Agent Reinforcement Learning (MARL). The MCA problem addressed how to distribute global reward, which is received by the Multi-Agent System (MAS) owing to an interaction with the environment among the agents. In this paper, a two-step method is proposed to solve the MCA using the bankruptcy and Evolutionary Games (EG). The first phase turns the MCA problem into a bankruptcy game through introducing the constraint of Maximum Performance Power (MPP). In this game, the agents act as players through a mixed strategy. The game’s outcome is the each agent’s share of the global reward. An instance of credit assignment is extracted by determining each agent’s share from the global reward. Using the mixed strategy leads to generate a lot of credit assignment instances. Therefore, finding the best instance of credit assignment is challenging. To select the best one in the second step, the EG is applied. In other words, the second step turns the MCA problem into an EG; the players are considered as the instances of credit assignment extracted from the first step. The proposed method was evaluated in an operational environment which is called Multi-score Puzzle (MsP) as compared to the state-of-the-art algorithms such as COMA, VDN, SQDDPG, ranking, dynamic and history-based methods. Simulation results indicated the better performance of the proposed method in terms of group learning rate, confidence, expertness, certainty and correctness. Efficiency was the only criterion in which the proposed method underperformed other methods.