Overview of the NTCIR-7 ACLIA IR4QA Task

This paper presents an overview of the IR4QA (In- formation Retrieval for Question Answering) Task of the NTCIR-7 ACLIA (Advanced Cross-lingual Infor- mation Access) Task Cluster. IR4QA evaluates tra- ditional ranked retrieval of documents using well- studied metrics such as Average Precision, but the re- trieval task is embedded in the context of cross-lingual question answering. That is, document retrieval is treated as a component of the entire question answer- ing system. This paper concentrates on how rele- vance assessments for the Simpified Chinese, Tradi- tional Chinese and Japanese IR4QA test collections were obtained, and the outcome of the formal IR4QA evaluation using the three collections. For the rela- tionship between IR4QA and the entire ACLIA task cluster, we refer the reader to the overview paper of ACLIA (17). For details of the individual IR4QA sys- tems, we refer the reader to the participants' reports.