Efficient Online Unlearning via Hessian-Free Recollection of Individual Data Statistics
arxiv(2024)
摘要
Machine unlearning strives to uphold the data owners' right to be forgotten
by enabling models to selectively forget specific data. Recent methods suggest
that one approach of data forgetting is by precomputing and storing statistics
carrying second-order information to improve computational and memory
efficiency. However, they rely on restrictive assumptions and the
computation/storage suffer from the curse of model parameter dimensionality,
making it challenging to apply to most deep neural networks. In this work, we
propose a Hessian-free online unlearning method. We propose to maintain a
statistical vector for each data point, computed through affine stochastic
recursion approximation of the difference between retrained and learned models.
Our proposed algorithm achieves near-instantaneous online unlearning as it only
requires a vector addition operation. Based on the strategy that recollecting
statistics for forgetting data, the proposed method significantly reduces the
unlearning runtime. Experimental studies demonstrate that the proposed scheme
surpasses existing results by orders of magnitude in terms of time and memory
costs, while also enhancing accuracy.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要