USimAgent: Large Language Models for Simulating Search Users
arxiv(2024)
摘要
Due to the advantages in the cost-efficiency and reproducibility, user
simulation has become a promising solution to the user-centric evaluation of
information retrieval systems. Nonetheless, accurately simulating user search
behaviors has long been a challenge, because users' actions in search are
highly complex and driven by intricate cognitive processes such as learning,
reasoning, and planning. Recently, Large Language Models (LLMs) have
demonstrated remarked potential in simulating human-level intelligence and have
been used in building autonomous agents for various tasks. However, the
potential of using LLMs in simulating search behaviors has not yet been fully
explored. In this paper, we introduce a LLM-based user search behavior
simulator, USimAgent. The proposed simulator can simulate users' querying,
clicking, and stopping behaviors during search, and thus, is capable of
generating complete search sessions for specific search tasks. Empirical
investigation on a real user behavior dataset shows that the proposed simulator
outperforms existing methods in query generation and is comparable to
traditional methods in predicting user clicks and stopping behaviors. These
results not only validate the effectiveness of using LLMs for user simulation
but also shed light on the development of a more robust and generic user
simulators.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要