InstaStyle: Inversion Noise of a Stylized Image is Secretly a Style Adviser
CoRR(2023)
摘要
Stylized text-to-image generation focuses on creating images from textual
descriptions while adhering to a style specified by a few reference images.
However, subtle style variations within different reference images can hinder
the model from accurately learning the target style. In this paper, we propose
InstaStyle, a novel approach that excels in generating high-fidelity stylized
images with only a single reference image. Our approach is based on the finding
that the inversion noise from a stylized reference image inherently carries the
style signal, as evidenced by their non-zero signal-to-noise ratio. We employ
DDIM inversion to extract this noise from the reference image and leverage a
diffusion model to generate new stylized images from the "style" noise.
Additionally, the inherent ambiguity and bias of textual prompts impede the
precise conveying of style. To address this, we introduce a learnable style
token via prompt refinement, which enhances the accuracy of the style
description for the reference image. Qualitative and quantitative experimental
results demonstrate that InstaStyle achieves superior performance compared to
current benchmarks. Furthermore, our approach also showcases its capability in
the creative task of style combination with mixed inversion noise.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要