Wiki-TabNER:Advancing Table Interpretation Through Named Entity Recognition
arxiv(2024)
摘要
Web tables contain a large amount of valuable knowledge and have inspired
tabular language models aimed at tackling table interpretation (TI) tasks. In
this paper, we analyse a widely used benchmark dataset for evaluation of TI
tasks, particularly focusing on the entity linking task. Our analysis reveals
that this dataset is overly simplified, potentially reducing its effectiveness
for thorough evaluation and failing to accurately represent tables as they
appear in the real-world. To overcome this drawback, we construct and annotate
a new more challenging dataset. In addition to introducing the new dataset, we
also introduce a novel problem aimed at addressing the entity linking task:
named entity recognition within cells. Finally, we propose a prompting
framework for evaluating the newly developed large language models (LLMs) on
this novel TI task. We conduct experiments on prompting LLMs under various
settings, where we use both random and similarity-based selection to choose the
examples presented to the models. Our ablation study helps us gain insights
into the impact of the few-shot examples. Additionally, we perform qualitative
analysis to gain insights into the challenges encountered by the models and to
understand the limitations of the proposed dataset.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要