MEASURING THE SIMILARITY OF TWO IMAGE SEQUENCES

We propose a novel similarity measure of two image sequences based on shapeme histograms. The idea of shapeme histogram has been used for single image/texture recognition, but is used here to solve the sequence-to- sequence matching problem. We develop techniques to represent each sequence as a set of shapeme histograms, which captures different variations of the object appear- ances within the sequence. These shapeme histograms are computed from the set of 2D invariant features that are sta- ble across multiple images in the sequence, and therefore minimizes the effect of both background clutter, and 2D pose variations. We define sequence similarity measure as the similarity of the most similar pair of images from both sequences. This definition maximizes the chance of match- ing between two sequences of the same object, because it requires only part of the sequences being similar. We also introduce a weighting scheme to conduct an implicit fea- ture selection process during the matching of two shapeme histograms. Experiments on clustering image sequences of tracked objects demonstrate the efficacy of the proposed method.