Simulation of pedestrian evacuation with reinforcement learning based on a dynamic scanning algorithm

Humans plan their movements mainly based on visual information. However, agents in few existing evacuation models perceive the environment by using visual information. To obtain the visual features of an individual in a discrete field, a Dynamic Scanning Algorithm (DSA) is proposed. DSA introduces the ray-scanning of LIDAR, a ”laser” is released by an agent to detect the nearest object that intersects it. By pre-storing the grids crossed by the rays, the efficiency of DSA is significantly improved. Using the scan results as inputs, an evacuation model has been developed based on the Deep Reinforcement Learning with Double Q-learning (DDQN). The parameters of DSA are calibrated at first, and a group of parameters with a good balance between efficiency and accuracy are recommended. Furthermore, the fundamental diagram is reproduced to calibrate the reward values in DDQN. At last, trajectories and behaviors in the model are studied by using the calibrated parameters. Results show that the movement trajectories are affected by visible distances and reward values, and some effectiveness are consistent with that in experiments. Besides, the exit selection behavior and the lane formation behavior are observed in simulation without introducing any special designed rules. DSA provides a new method to obtain the first-person environmental information in discrete field, and the DSA&DDQN based model defines a new ray-scan-based evacuation modeling scheme.