FedADMM-InSa: An Inexact and Self-Adaptive ADMM for Federated Learning
CoRR(2024)
摘要
Federated learning (FL) is a promising framework for learning from
distributed data while maintaining privacy. The development of efficient FL
algorithms encounters various challenges, including heterogeneous data and
systems, limited communication capacities, and constrained local computational
resources. Recently developed FedADMM methods show great resilience to both
data and system heterogeneity. However, they still suffer from performance
deterioration if the hyperparameters are not carefully tuned. To address this
issue, we propose an inexact and self-adaptive FedADMM algorithm, termed
FedADMM-InSa. First, we design an inexactness criterion for the clients' local
updates to eliminate the need for empirically setting the local training
accuracy. This inexactness criterion can be assessed by each client
independently based on its unique condition, thereby reducing the local
computational cost and mitigating the undesirable straggle effect. The
convergence of the resulting inexact ADMM is proved under the assumption of
strongly convex loss functions. Additionally, we present a self-adaptive scheme
that dynamically adjusts each client's penalty parameter, enhancing algorithm
robustness by mitigating the need for empirical penalty parameter choices for
each client. Extensive numerical experiments on both synthetic and real-world
datasets are conducted. As validated by some numerical tests, our proposed
algorithm can reduce the clients' local computational load significantly and
also accelerate the learning process compared to the vanilla FedADMM.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要