GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 32 (NIPS 2019), 2019.
Scaling up deep neural network capacity has been known as an effective approach to improving model quality for several different machine learning tasks. In many cases, increasing model capacity beyond the memory limit of a single accelerator has required developing special algorithms or infrastructure. These solutions are often architectu...More
Full Text (Upload PDF)
PPT (Upload PPT)