Scientific Opinion Summarization: Paper Meta-review Generation Dataset, Methods, and Evaluation
arxiv(2023)
摘要
Opinions in scientific research papers can be divergent, leading to
controversies among reviewers. However, most existing datasets for opinion
summarization are centered around product reviews and assume that the analyzed
opinions are non-controversial, failing to account for the variability seen in
other contexts such as academic papers, political debates, or social media
discussions. To address this gap, we propose the task of scientific opinion
summarization, where research paper reviews are synthesized into meta-reviews.
To facilitate this task, we introduce the ORSUM dataset covering 15,062 paper
meta-reviews and 57,536 paper reviews from 47 conferences. Furthermore, we
propose the Checklist-guided Iterative Introspection approach, which breaks
down scientific opinion summarization into several stages, iteratively refining
the summary under the guidance of questions from a checklist. Our experiments
show that (1) human-written summaries do not always satisfy all necessary
criteria such as depth of discussion, and identifying consensus and controversy
for the specific domain, and (2) the combination of task decomposition and
iterative self-refinement shows strong potential for enhancing the opinions and
can be applied to other complex text generation using black-box LLMs.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要