Fed-CVLC: Compressing Federated Learning Communications with Variable-Length Codes
CoRR(2024)
摘要
In Federated Learning (FL) paradigm, a parameter server (PS) concurrently
communicates with distributed participating clients for model collection,
update aggregation, and model distribution over multiple rounds, without
touching private data owned by individual clients. FL is appealing in
preserving data privacy; yet the communication between the PS and scattered
clients can be a severe bottleneck. Model compression algorithms, such as
quantization and sparsification, have been suggested but they generally assume
a fixed code length, which does not reflect the heterogeneity and variability
of model updates. In this paper, through both analysis and experiments, we show
strong evidences that variable-length is beneficial for compression in FL. We
accordingly present Fed-CVLC (Federated Learning Compression with
Variable-Length Codes), which fine-tunes the code length in response of the
dynamics of model updates. We develop optimal tuning strategy that minimizes
the loss function (equivalent to maximizing the model utility) subject to the
budget for communication. We further demonstrate that Fed-CVLC is indeed a
general compression design that bridges quantization and sparsification, with
greater flexibility. Extensive experiments have been conducted with public
datasets to demonstrate that Fed-CVLC remarkably outperforms state-of-the-art
baselines, improving model utility by 1.50
traffic by 16.67
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要