Impact of item consumption on assessment of recommendations in user studies.
RecSys '18: Twelfth ACM Conference on Recommender Systems Vancouver British Columbia Canada October, 2018(2018)
摘要
In user studies of recommender systems, participants typically cannot consume the recommended items. Still, they are asked to assess recommendation quality and other aspects related to user experience by means of questionnaires. Without having listened to recommended songs or watched suggested movies, however, this might be an error-prone task, possibly limiting validity of results obtained in these studies. In this paper, we investigate the effect of actually consuming the recommended items. We present two user studies conducted in different domains showing that in some cases, differences in the assessment of recommendations and in questionnaire results occur. Apparently, it is not always possible to adequately measure user experience without allowing users to consume items. On the other hand, depending on domain and provided information, participants sometimes seem to approximate the actual value of recommendations reasonably well.
更多查看译文
关键词
Recommender Systems, Experimentation, User Studies
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络