PathoTune: Adapting Visual Foundation Model to Pathological Specialists
arxiv(2024)
摘要
As natural image understanding moves towards the pretrain-finetune era,
research in pathology imaging is concurrently evolving. Despite the predominant
focus on pretraining pathological foundation models, how to adapt foundation
models to downstream tasks is little explored. For downstream adaptation, we
propose the existence of two domain gaps, i.e., the Foundation-Task Gap and the
Task-Instance Gap. To mitigate these gaps, we introduce PathoTune, a framework
designed to efficiently adapt pathological or even visual foundation models to
pathology-specific tasks via multi-modal prompt tuning. The proposed framework
leverages Task-specific Visual Prompts and Task-specific Textual Prompts to
identify task-relevant features, along with Instance-specific Visual Prompts
for encoding single pathological image features. Results across multiple
datasets at both patch-level and WSI-level demonstrate its superior performance
over single-modality prompt tuning approaches. Significantly, PathoTune
facilitates the direct adaptation of natural visual foundation models to
pathological tasks, drastically outperforming pathological foundation models
with simple linear probing. The code will be available upon acceptance.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要