A motional but temporally consistent physical video examples

Adversarial examples (AEs) attract extensive attention due to their inherent security-related properties of attacking Deep Neural Networks (DNNs) through carefully constructed modifications. Recently, they have been extended to video tasks. Human action recognition based on DNNs is a crucial task in video tasks. AEs of human action recognition models attract much focus and previous works demonstrated the vulnerability of AEs of human action recognition in the digital world. However, the adversarial videos for attacking human action recognition models in the physical world are still in the open stage. The design of physical adversarial videos is crucial for helping to evaluate the robustness of some critical applications based on human action recognition, e.g., surveillance and pedestrian detection. Unlike the digital attacks on action recognition models, the perturbations of physical adversarial videos should be motional but temporally consistent across each whole video. The attacks need to destroy the spatial interactions and temporal interactions in videos. Previously developed attacks for video models in the digital world are difficult to transfer to the physical world. In this paper, we close this gap, and we are the first to attack human action recognition models in the physical world. We first generate a dynamic mask via an improved object tracking method and then use the center location to construct a motion location map. Finally, gradient sharing method is used to generate temporally consistent perturbations and optimize the perturbations into robust patches. Experiments show that these patches can successfully attack a real-time human action recognition system, and the proposed approach has a 77.5% success rate in this setting.