UnlearnCanvas: A Stylized Image Dataset to Benchmark Machine Unlearning for Diffusion Models
CoRR(2024)
摘要
The rapid advancement of diffusion models (DMs) has not only transformed
various real-world industries but has also introduced negative societal
concerns, including the generation of harmful content, copyright disputes, and
the rise of stereotypes and biases. To mitigate these issues, machine
unlearning (MU) has emerged as a potential solution, demonstrating its ability
to remove undesired generative capabilities of DMs in various applications.
However, by examining existing MU evaluation methods, we uncover several key
challenges that can result in incomplete, inaccurate, or biased evaluations for
MU in DMs. To address them, we enhance the evaluation metrics for MU, including
the introduction of an often-overlooked retainability measurement for DMs
post-unlearning. Additionally, we introduce UnlearnCanvas, a comprehensive
high-resolution stylized image dataset that facilitates us to evaluate the
unlearning of artistic painting styles in conjunction with associated image
objects. We show that this dataset plays a pivotal role in establishing a
standardized and automated evaluation framework for MU techniques on DMs,
featuring 7 quantitative metrics to address various aspects of unlearning
effectiveness. Through extensive experiments, we benchmark 5 state-of-the-art
MU methods, revealing novel insights into their pros and cons, and the
underlying unlearning mechanisms. Furthermore, we demonstrate the potential of
UnlearnCanvas to benchmark other generative modeling tasks, such as style
transfer. The UnlearnCanvas dataset, benchmark, and the codes to reproduce all
the results in this work can be found at
https://github.com/OPTML-Group/UnlearnCanvas.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要