Consprompt: Exploiting Contrastive Samples for Few-Shot Prompt Learning

Prompt has become an effective linguistic tool for utilizing pre-trained language models. However, in few-shot scenarios, subtle changes of prompt’s design always make the result widely different, and the prompt learning methods are also easy to overfit the limited samples. To alleviate this, we explore utilizing suitable contrastive samples and multi-degree contrastive learning methods to improve the robustness of prompt’s representation. Therefore, the proposed Consprompt combined with prompt encoding network, contrastive sampling modules, and contrastive scoring modules, is introduced to realize differential contrastive learning. Our results exhibit the state-of-the-art performance in different few-shot settings, and the ablation experiments also certify the effectiveness of utilizing multi-degree contrastive learning in prompt-based fine-tuning process.