Evaluating Information Retrieval Systems Under The Challenges Of Interaction And Multidimensional Dynamic Relevance
COLIS4: EMERGING FRAMEWORKS AND METHODS(2002)
摘要
The Laboratory Model of information retrieval (IR) evaluation has been challenged by progress in research related to relevance and information seeking as well as by the growing need for accounting for interaction in evaluation. Real human users introduce non-binary, subjective and dynamic relevance judgments into IR processes and affect these processes. Therefore the traditional evaluation based on the Laboratory Model is challenged for its (lack of) realism. This paper examines the rationale of evaluating the IR algorithms, the status of the traditional evaluation, and the applicability of the proposed novel evaluation methods and measures. It further points out research problems requiring attention for further advances in the area. The Laboratory Model is found limited but still useful for the specific tasks it fulfills in the development of IR algorithms.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络