On the Reliability of User Feedback for Evaluating the Quality of Conversational Agents
PROCEEDINGS OF THE 32ND ACM INTERNATIONAL CONFERENCE ON INFORMATION AND KNOWLEDGE MANAGEMENT, CIKM 2023(2023)
摘要
We analyse the reliability of users' explicit feedback for evaluating the quality of conversational agents. Using data from a commercial conversational system, we analyse how user feedback compares with human annotations; how well it aligns with implicit user satisfaction signals, such as retention; and how much user feedback is needed to reliably evaluate the quality of a conversational system.
更多查看译文
关键词
conversational agent quality,user feedback reliability
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要