MAIRA-1: A specialised large multimodal model for radiology report generation
CoRR(2023)
摘要
We present a radiology-specific multimodal model for the task for generating
radiological reports from chest X-rays (CXRs). Our work builds on the idea that
large language model(s) can be equipped with multimodal capabilities through
alignment with pre-trained vision encoders. On natural images, this has been
shown to allow multimodal models to gain image understanding and description
capabilities. Our proposed model (MAIRA-1) leverages a CXR-specific image
encoder in conjunction with a fine-tuned large language model based on
Vicuna-7B, and text-based data augmentation, to produce reports with
state-of-the-art quality. In particular, MAIRA-1 significantly improves on the
radiologist-aligned RadCliQ metric and across all lexical metrics considered.
Manual review of model outputs demonstrates promising fluency and accuracy of
generated reports while uncovering failure modes not captured by existing
evaluation practices. More information and resources can be found on the
project website: https://aka.ms/maira.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要