On encoding temporal evolution for real-time action prediction

Anticipating future actions is a key component ofintelligence, specifically when it applies to realtimesystems, such as robots or autonomous cars.While recent works have addressed prediction ofraw RGB pixel values, we focus on anticipating themotion evolution in future video frames. To thisend, we construct dynamic images (DIs) by summarisingmoving pixels through a sequence of futureframes. We train a convolutional LSTMs topredict the next DIs based on an unsupervisedlearning process, and then recognise the activity associatedwith the predicted DI. We demonstrate theeffectiveness of our approach on 3 benchmark actiondatasets showing that despite running on videoswith complex activities, our approach is able toanticipate the next human action with high accuracyand obtain better results than the state-of-the-artmethods.