Understanding the Capabilities and Limitations of Large Language Models for Cultural Commonsense
arxiv(2024)
摘要
Large language models (LLMs) have demonstrated substantial commonsense
understanding through numerous benchmark evaluations. However, their
understanding of cultural commonsense remains largely unexamined. In this
paper, we conduct a comprehensive examination of the capabilities and
limitations of several state-of-the-art LLMs in the context of cultural
commonsense tasks. Using several general and cultural commonsense benchmarks,
we find that (1) LLMs have a significant discrepancy in performance when tested
on culture-specific commonsense knowledge for different cultures; (2) LLMs'
general commonsense capability is affected by cultural context; and (3) The
language used to query the LLMs can impact their performance on
cultural-related tasks. Our study points to the inherent bias in the cultural
understanding of LLMs and provides insights that can help develop culturally
aware language models.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要