SciGLM: Training Scientific Language Models with Self-Reflective Instruction Annotation and Tuning
CoRR(2024)
摘要
Large Language Models (LLMs) have shown promise in
assisting scientific discovery. However, such applications are currently
limited by LLMs' deficiencies in understanding intricate scientific concepts,
deriving symbolic equations, and solving advanced numerical calculations. To
bridge these gaps, we introduce SciGLM, a suite of scientific language models
able to conduct college-level scientific reasoning. Central to our approach is
a novel self-reflective instruction annotation framework to address the data
scarcity challenge in the science domain. This framework leverages existing
LLMs to generate step-by-step reasoning for unlabelled scientific questions,
followed by a process of self-reflective critic-and-revise. Applying this
framework, we curated SciInstruct, a diverse and high-quality dataset
encompassing mathematics, physics, chemistry, and formal proofs. We fine-tuned
the ChatGLM family of language models with SciInstruct, enhancing their
capabilities in scientific and mathematical reasoning. Remarkably, SciGLM
consistently improves both the base model (ChatGLM3-6B-Base) and larger-scale
models (12B and 32B), without sacrificing the language understanding
capabilities of the base model. This makes SciGLM a suitable foundational model
to facilitate diverse scientific discovery tasks. For the benefit of the wider
research community, we release SciInstruct, SciGLM, alongside a self-reflective
framework and fine-tuning code at .
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要