Investigating Learning Methods and Environment Representation in the Construction of Player Agents: Application on FIFA Game

The objective behind this study is to investigate Machine Learning (ML) techniques combined with methods from Computer Vision (CV) for state representation by images, to produce agents capable of solving problems, in real time, in environments with complex properties. Such difficulties require agents to be highly efficient in their learning (and, consequently, decision-making) and environmental perception processes, without which they will not be successful. The digital game FIFA - soccer simulator - is used as a case study because it represents a realistic and challenging environment. The ML techniques are investigated in the context of the Deep Learning (DL) approach provided by Convolutional Neural Networks (CNNs), being: imitation learning, used here with the purpose of endowing the agent with the ability to solve problems in a way closer to human; by deep reinforcement, in which the agent is trained in an attempt to autonomously abstract an optimal decision-making policy. Regarding the environmental perception, the following state representations approaches are investigated in this study: raw images - with and without color information - and through Object Detection Techniques (ODT). In order to further improve the performance of the agents produced, genetic algorithm techniques are explored to automatically define a CNN architecture to be used as the player agents decision-making module. In addition to corroborating the excellent results that DL combined with CV has been producing in the context of ML (particularly in games), the present work shows the great potential of the application of ODT in the process of enhancing the environmental perception, which counts as a relevant counterpart to the fact that ODT demands computational procedures with a higher cost in relation to the representations based on raw images.