A two-generation based method for few-shot learning with few-shot instance-level privileged information

Few-shot Learning (FSL) aims to recognize the novel classes from few novel samples. Recently, lots of methods have been proposed to improve FSL performance by introducing privileged information. However, on the one hand, they utilize the class name or class-level description generated by some tools such as WordNet as the privileged information. On the other hand, they are all one-generation based and just use the simple convex integration of visual modality and privileged information modality. Besides, the classic FSL dataset miniImageNet has no labels for few-shot instance-level privileged information. In this paper, we propose that the few-shot instance-level privileged information generated by few-shot visual images samples are more concrete and more diverse, which is more in line with the real world situation than the class-level privileged information that is the abstract concept summarized from a large number of visual image samples. For this, we propose a novel Two-generation based FSL method (2G-FSL) which transfers the prior knowledge from the prior model to the posterior model. This can make 2G-FSL learns the meta-knowledge about preserving correct prior knowledge and self-correcting erroneous prior knowledge after introducing the few-shot instance-level privileged information, growing into a more robust posterior model. In 2G-FSL, we introduce a novel Latent Feature Augmentation (LFA) module in posterior model to learn the episode-related augmentation and integration of the latent features of visual and privileged information modalities instead of the simple convex integration, which can generate diverse modality integration strategies for enhancing the diversity of latent features to make the features more robust to alleviate the insufficient data problem of FSL. We make the dataset of few-shot instance-level privileged information of miniImageNet publicly available for the subsequent research of FSL with few-shot instance-level privileged information. Experimental results demonstrate the effectiveness and superiority of 2G-FSL with LFA in FSL with few-shot instance-level privileged information.