Iterative Prompt Refinement for Radiation Oncology Symptom Extraction Using Teacher-Student Large Language Models
CoRR(2024)
摘要
This study introduces a novel teacher-student architecture utilizing Large
Language Models (LLMs) to improve prostate cancer radiotherapy symptom
extraction from clinical notes. Mixtral, the student model, initially extracts
symptoms, followed by GPT-4, the teacher model, which refines prompts based on
Mixtral's performance. This iterative process involved 294 single symptom
clinical notes across 12 symptoms, with up to 16 rounds of refinement per
epoch. Results showed significant improvements in extracting symptoms from both
single and multi-symptom notes. For 59 single symptom notes, accuracy increased
from 0.51 to 0.71, precision from 0.52 to 0.82, recall from 0.52 to 0.72, and
F1 score from 0.49 to 0.73. In 375 multi-symptom notes, accuracy rose from 0.24
to 0.43, precision from 0.6 to 0.76, recall from 0.24 to 0.43, and F1 score
from 0.20 to 0.44. These results demonstrate the effectiveness of advanced
prompt engineering in LLMs for radiation oncology use.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要