MoE-FFD: Mixture of Experts for Generalized and Parameter-Efficient Face Forgery Detection
arxiv(2024)
摘要
Deepfakes have recently raised significant trust issues and security concerns
among the public. Compared to CNN face forgery detectors, ViT-based methods
take advantage of the expressivity of transformers, achieving superior
detection performance. However, these approaches still exhibit the following
limitations: (1). Fully fine-tuning ViT-based models from ImageNet weights
demands substantial computational and storage resources; (2). ViT-based methods
struggle to capture local forgery clues, leading to model bias and limited
generalizability. To tackle these challenges, this work introduces
Mixture-of-Experts modules for Face Forgery Detection (MoE-FFD), a generalized
yet parameter-efficient ViT-based approach. MoE-FFD only updates lightweight
Low-Rank Adaptation (LoRA) and Adapter layers while keeping the ViT backbone
frozen, thereby achieving parameter-efficient training. Moreover, MoE-FFD
leverages the expressivity of transformers and local priors of CNNs to
simultaneously extract global and local forgery clues. Additionally, novel MoE
modules are designed to scale the model's capacity and select optimal forgery
experts, further enhancing forgery detection performance. The proposed MoE
learning scheme can be seamlessly adapted to various transformer backbones in a
plug-and-play manner. Extensive experimental results demonstrate that the
proposed method achieves state-of-the-art face forgery detection performance
with reduced parameter overhead. The code will be released upon acceptance.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要