How the Advent of Ubiquitous Large Language Models both Stymie and Turbocharge Dynamic Adversarial Question Generation
CoRR(2024)
摘要
Dynamic adversarial question generation, where humans write examples to stump
a model, aims to create examples that are realistic and informative. However,
the advent of large language models (LLMs) has been a double-edged sword for
human authors: more people are interested in seeing and pushing the limits of
these models, but because the models are so much stronger an opponent, they are
harder to defeat. To understand how these models impact adversarial question
writing process, we enrich the writing guidance with LLMs and retrieval models
for the authors to reason why their questions are not adversarial. While
authors could create interesting, challenging adversarial questions, they
sometimes resort to tricks that result in poor questions that are ambiguous,
subjective, or confusing not just to a computer but also to humans. To address
these issues, we propose new metrics and incentives for eliciting good,
challenging questions and present a new dataset of adversarially authored
questions.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要