Pcpcad: Proposal Complementary Action Detector

Temporal action detection is still a challenging task. This task not only requires correct classification, but also needs to accurately detect the start and end times of each action. In this paper, we present a novel proposal complementary action detector (PCAD) to deal with video streams under continuous, untrimmed conditions. Our approach first uses a simple fully 3D convolutional (Conv3D) network to encode the video streams and then generates candidate temporal proposals for activities by using anchor segments. To generate more precise proposals, we also designed a boundary proposal network (BPN) to offer some complementary information for the candidate proposals. Finally, we learn an efficient classifier to classify the generated proposals into different activities and refine their temporal boundaries at the same time. Our model can achieve end-to-end training by jointly optimizing classification loss and regression loss. When evaluating on THUMOS'14 detection benchmark, PCAD achieves the state-of-the-art performance in high-speed models.