In-context Learning with Retrieved Demonstrations for Language Models: A Survey
CoRR(2024)
摘要
Language models, especially pre-trained large language models, have showcased
remarkable abilities as few-shot in-context learners (ICL), adept at adapting
to new tasks with just a few demonstrations in the input context. However, the
model's ability to perform ICL is sensitive to the choice of the few-shot
demonstrations. Instead of using a fixed set of demonstrations, one recent
development is to retrieve demonstrations tailored to each input query. The
implementation of demonstration retrieval is relatively straightforward,
leveraging existing databases and retrieval systems. This not only improves the
efficiency and scalability of the learning process but also has been shown to
reduce biases inherent in manual example selection. In light of the encouraging
results and growing research in ICL with retrieved demonstrations, we conduct
an extensive review of studies in this area. In this survey, we discuss and
compare different design choices for retrieval models, retrieval training
procedures, and inference algorithms.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要