Scene Graph Aided Radiology Report Generation
arxiv(2024)
摘要
Radiology report generation (RRG) methods often lack sufficient medical
knowledge to produce clinically accurate reports. The scene graph contains rich
information to describe the objects in an image. We explore enriching the
medical knowledge for RRG via a scene graph, which has not been done in the
current RRG literature. To this end, we propose the Scene Graph aided RRG
(SGRRG) network, a framework that generates region-level visual features,
predicts anatomical attributes, and leverages an automatically generated scene
graph, thus achieving medical knowledge distillation in an end-to-end manner.
SGRRG is composed of a dedicated scene graph encoder responsible for
translating the scene graph, and a scene graph-aided decoder that takes
advantage of both patch-level and region-level visual information. A
fine-grained, sentence-level attention method is designed to better dis-till
the scene graph information. Extensive experiments demonstrate that SGRRG
outperforms previous state-of-the-art methods in report generation and can
better capture abnormal findings.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要