Towards Human Activity Reasoning With Computational Logic And Deep Learning

We approach the problem of human action recognition in videos by distinguishing between simple and complex actions. To recognize simple actions, we take advantage of the latest advances with 3D convolutional networks, which are able to offer a generic video snippet descriptor. For the complex ones, which involve interaction between more than one individual, we use the recognized simple human actions of the previous step to generate Event Calculus theories. This way, we aim to achieve a high-level human action understanding, combining the opaque effectiveness of deep learning and the transparent reasoning of computational logic. Our experimental results on a benchmark activity recognition dataset encourage further research towards this direction.