MLaKE: Multilingual Knowledge Editing Benchmark for Large Language Models
arxiv(2024)
摘要
The extensive utilization of large language models (LLMs) underscores the
crucial necessity for precise and contemporary knowledge embedded within their
intrinsic parameters. Existing research on knowledge editing primarily
concentrates on monolingual scenarios, neglecting the complexities presented by
multilingual contexts and multi-hop reasoning. To address these challenges, our
study introduces MLaKE (Multilingual Language Knowledge Editing), a novel
benchmark comprising 4072 multi-hop and 5360 single-hop questions designed to
evaluate the adaptability of knowledge editing methods across five languages:
English, Chinese, Japanese, French, and German. MLaKE aggregates fact chains
from Wikipedia across languages and utilizes LLMs to generate questions in both
free-form and multiple-choice. We evaluate the multilingual knowledge editing
generalization capabilities of existing methods on MLaKE. Existing knowledge
editing methods demonstrate higher success rates in English samples compared to
other languages. However, their generalization capabilities are limited in
multi-language experiments. Notably, existing knowledge editing methods often
show relatively high generalization for languages within the same language
family compared to languages from different language families. These results
underscore the imperative need for advancements in multilingual knowledge
editing and we hope MLaKE can serve as a valuable resource for benchmarking and
solution development.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要