Simultaneous Tri-Modal Medical Image Fusion and Super-Resolution using Conditional Diffusion Model
arxiv(2024)
摘要
In clinical practice, tri-modal medical image fusion, compared to the
existing dual-modal technique, can provide a more comprehensive view of the
lesions, aiding physicians in evaluating the disease's shape, location, and
biological activity. However, due to the limitations of imaging equipment and
considerations for patient safety, the quality of medical images is usually
limited, leading to sub-optimal fusion performance, and affecting the depth of
image analysis by the physician. Thus, there is an urgent need for a technology
that can both enhance image resolution and integrate multi-modal information.
Although current image processing methods can effectively address image fusion
and super-resolution individually, solving both problems synchronously remains
extremely challenging. In this paper, we propose TFS-Diff, a simultaneously
realize tri-modal medical image fusion and super-resolution model. Specially,
TFS-Diff is based on the diffusion model generation of a random iterative
denoising process. We also develop a simple objective function and the proposed
fusion super-resolution loss, effectively evaluates the uncertainty in the
fusion and ensures the stability of the optimization process. And the channel
attention module is proposed to effectively integrate key information from
different modalities for clinical diagnosis, avoiding information loss caused
by multiple image processing. Extensive experiments on public Harvard datasets
show that TFS-Diff significantly surpass the existing state-of-the-art methods
in both quantitative and visual evaluations. The source code will be available
at GitHub.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要