Scene Classification With Semantic Fisher Vectors
2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)(2015)
摘要
With the help of a convolutional neural network (CNN) trained to recognize objects, a scene image is represented as a hag of semantics (BoS). This involves classifying image patches using the network and considering the class posterior probability vectors as locally extracted semantic descriptors. The image BoS is summarized using a Fisher vector (EV) embedding that exploits the properties of the space of these descriptors. The resulting representation is referred to as a semantic Fisher vector Two implementations of a semantic EV are investigated. First involves modeling the BoS with a Dirichlet Mixture and computing the Fisher gradients for this model. Due to the difficulty of mixture modeling on a non-Euclidean probability simplex this approach is shown to be unsuccessful. A second implementation is derived using the interpretation of semantic descriptors as parameters of a multinomial distribution. Like the parameters of any exponential family, these can he projected into their natural parameter space. For a CNN, this is shown equivalent to using inputs of its soft-max layer as patch descriptors. A semantic FV is then computed as a Gaussian Mixture FV in the space of these natural parameters. This representation is shown to outperform other alternatives such as FVs of features from the intermediate CNN layers or a classifier obtained by adapting (fine-tuning) the CNN. The proposed EV represents an embedding for object classification probabilities. As an image representation, therefore, it is complementary to the features obtained from a scene classification CNN. A combination of the two representations is shown to achieve state-of-the-art results on MIT Indoor scenes and SUN datasets.
更多查看译文
关键词
scene classification,semantic Fisher vectors,convolutional neural network,CNN,object recognition,scene image,image representation,bag of semantics,BoS,image patches,class posterior probability vectors,Dirichlet mixture,Fisher gradients,non-Euclidean probability simplex,multinomial distribution,object classification probabilities
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络