Snake Learning: A Communication- and Computation-Efficient Distributed Learning Framework for 6G
arxiv(2024)
摘要
In the evolution towards 6G, integrating Artificial Intelligence (AI) with
advanced network infrastructure emerges as a pivotal strategy for enhancing
network intelligence and resource utilization. Existing distributed learning
frameworks like Federated Learning and Split Learning often struggle with
significant challenges in dynamic network environments including high
synchronization demands, costly communication overheads, severe computing
resource consumption, and data heterogeneity across network nodes. These
obstacles hinder the applications of ubiquitous computing capabilities of 6G
networks, especially in light of the trend of escalating model parameters and
training data volumes. To address these challenges effectively, this paper
introduces "Snake Learning", a cost-effective distributed learning framework.
Specifically, Snake Learning respects the heterogeneity of inter-node computing
capability and local data distribution in 6G networks, and sequentially trains
the designated part of model layers on individual nodes. This layer-by-layer
serpentine update mechanism contributes to significantly reducing the
requirements for storage, memory and communication during the model training
phase, and demonstrates superior adaptability and efficiency for both Computer
Vision (CV) training and Large Language Model (LLM) fine-tuning tasks across
homogeneous and heterogeneous data distributions.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要