Separate-and-Enhance: Compositional Finetuning for Text2Image Diffusion Models
CoRR(2023)
摘要
Despite recent significant strides achieved by diffusion-based Text-to-Image
(T2I) models, current systems are still less capable of ensuring decent
compositional generation aligned with text prompts, particularly for the
multi-object generation. This work illuminates the fundamental reasons for such
misalignment, pinpointing issues related to low attention activation scores and
mask overlaps. While previous research efforts have individually tackled these
issues, we assert that a holistic approach is paramount. Thus, we propose two
novel objectives, the Separate loss and the Enhance loss, that reduce object
mask overlaps and maximize attention scores, respectively. Our method diverges
from conventional test-time-adaptation techniques, focusing on finetuning
critical parameters, which enhances scalability and generalizability.
Comprehensive evaluations demonstrate the superior performance of our model in
terms of image realism, text-image alignment, and adaptability, notably
outperforming prominent baselines. Ultimately, this research paves the way for
T2I diffusion models with enhanced compositional capacities and broader
applicability.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要