Revisiting Automatic Question Summarization Evaluation in the Biomedical Domain

arxiv(2023)

引用 0|浏览36
暂无评分
摘要
Automatic evaluation metrics have been facilitating the rapid development of automatic summarization methods by providing instant and fair assessments of the quality of summaries. Most metrics have been developed for the general domain, especially news and meeting notes, or other language-generation tasks. However, these metrics are applied to evaluate summarization systems in different domains, such as biomedical question summarization. To better understand whether commonly used evaluation metrics are capable of evaluating automatic summarization in the biomedical domain, we conduct human evaluations of summarization quality from four different aspects of a biomedical question summarization task. Based on human judgments, we identify different noteworthy features for current automatic metrics and summarization systems as well. We also release a dataset of our human annotations to aid the research of summarization evaluation metrics in the biomedical domain.
更多
查看译文
关键词
automatic question summarization evaluation,biomedical
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络