DP-RDM: Adapting Diffusion Models to Private Domains Without Fine-Tuning
arxiv(2024)
摘要
Text-to-image diffusion models have been shown to suffer from sample-level
memorization, possibly reproducing near-perfect replica of images that they are
trained on, which may be undesirable. To remedy this issue, we develop the
first differentially private (DP) retrieval-augmented generation algorithm that
is capable of generating high-quality image samples while providing provable
privacy guarantees. Specifically, we assume access to a text-to-image diffusion
model trained on a small amount of public data, and design a DP retrieval
mechanism to augment the text prompt with samples retrieved from a private
retrieval dataset. Our differentially private retrieval-augmented
diffusion model (DP-RDM) requires no fine-tuning on the retrieval dataset to
adapt to another domain, and can use state-of-the-art generative models to
generate high-quality image samples while satisfying rigorous DP guarantees.
For instance, when evaluated on MS-COCO, our DP-RDM can generate samples with a
privacy budget of ϵ=10, while providing a 3.5 point improvement in
FID compared to public-only retrieval for up to 10,000 queries.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要