Shallow Synthesis of Knowledge in GPT-Generated Texts: A Case Study in Automatic Related Work Composition
CoRR(2024)
摘要
Numerous AI-assisted scholarly applications have been developed to aid
different stages of the research process. We present an analysis of AI-assisted
scholarly writing generated with ScholaCite, a tool we built that is designed
for organizing literature and composing Related Work sections for academic
papers. Our evaluation method focuses on the analysis of citation graphs to
assess the structural complexity and inter-connectedness of citations in texts
and involves a three-way comparison between (1) original human-written texts,
(2) purely GPT-generated texts, and (3) human-AI collaborative texts. We find
that GPT-4 can generate reasonable coarse-grained citation groupings to support
human users in brainstorming, but fails to perform detailed synthesis of
related works without human intervention. We suggest that future writing
assistant tools should not be used to draft text independently of the human
author.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要