FLASH: Federated Learning Across Simultaneous Heterogeneities
CoRR(2024)
摘要
The key premise of federated learning (FL) is to train ML models across a
diverse set of data-owners (clients), without exchanging local data. An
overarching challenge to this date is client heterogeneity, which may arise not
only from variations in data distribution, but also in data quality, as well as
compute/communication latency. An integrated view of these diverse and
concurrent sources of heterogeneity is critical; for instance, low-latency
clients may have poor data quality, and vice versa. In this work, we propose
FLASH(Federated Learning Across Simultaneous Heterogeneities), a lightweight
and flexible client selection algorithm that outperforms state-of-the-art FL
frameworks under extensive sources of heterogeneity, by trading-off the
statistical information associated with the client's data quality, data
distribution, and latency. FLASH is the first method, to our knowledge, for
handling all these heterogeneities in a unified manner. To do so, FLASH models
the learning dynamics through contextual multi-armed bandits (CMAB) and
dynamically selects the most promising clients. Through extensive experiments,
we demonstrate that FLASH achieves substantial and consistent improvements over
state-of-the-art baselines – as much as 10
its unified approach. Importantly, FLASH also outperforms federated aggregation
methods that are designed to handle highly heterogeneous settings and even
enjoys a performance boost when integrated with them.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要