A study on sampling strategies in space-time domain for recognition applications

We investigate the relative strengths of existing space-time interest points in the context of action detection and recognition. The interest point operators evaluated are an extension of the Harris corner detector (Laptev et al. [1]), a space-time Gabor filter (Dollar et al. [2]), and randomized sampling on the motion boundaries. In the first level of experiments we study the low level attributes of interest points such as stability, repeatability and sparsity with respect to the sources of variations such as actors, viewpoint and action category. In the second level we measure the discriminative power of interest points by extracting generic region descriptors around the interest points (1. histogram of optical flow[3], 2. motion history images[4], 3. histograms of oriented gradients[3]). Then we build a simple action recognition scheme by constructing a dictionary of codewords and learning a recognition system using the histograms of these codewords. We demonstrate that although there may be merits due to the structural information contained in the interest point detections, ultimately getting as many data samples as possible, even with random sampling, is the decisive factor in the interpretation of space-time data.