Flashback: Understanding and Mitigating Forgetting in Federated Learning
CoRR(2024)
摘要
In Federated Learning (FL), forgetting, or the loss of knowledge across
rounds, hampers algorithm convergence, particularly in the presence of severe
data heterogeneity among clients. This study explores the nuances of this
issue, emphasizing the critical role of forgetting in FL's inefficient learning
within heterogeneous data contexts. Knowledge loss occurs in both client-local
updates and server-side aggregation steps; addressing one without the other
fails to mitigate forgetting. We introduce a metric to measure forgetting
granularly, ensuring distinct recognition amid new knowledge acquisition.
Leveraging these insights, we propose Flashback, an FL algorithm with a dynamic
distillation approach that is used to regularize the local models, and
effectively aggregate their knowledge. Across different benchmarks, Flashback
outperforms other methods, mitigates forgetting, and achieves faster
round-to-target-accuracy, by converging in 6 to 16 rounds.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要