Soft Mixture Denoising: Beyond the Expressive Bottleneck of Diffusion Models
ICLR 2024(2023)
摘要
Because diffusion models have shown impressive performances in a number of
tasks, such as image synthesis, there is a trend in recent works to prove (with
certain assumptions) that these models have strong approximation capabilities.
In this paper, we show that current diffusion models actually have an
expressive bottleneck in backward denoising and some assumption made by
existing theoretical guarantees is too strong. Based on this finding, we prove
that diffusion models have unbounded errors in both local and global denoising.
In light of our theoretical studies, we introduce soft mixture denoising (SMD),
an expressive and efficient model for backward denoising. SMD not only permits
diffusion models to well approximate any Gaussian mixture distributions in
theory, but also is simple and efficient for implementation. Our experiments on
multiple image datasets show that SMD significantly improves different types of
diffusion models (e.g., DDPM), espeically in the situation of few backward
iterations.
更多查看译文
关键词
Diffusion Models,Expressive Bottleneck,Soft Mixture Denoising (SMD)
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要