GLaM: Efficient Scaling of Language Models with Mixture-of-Experts
GSPMD: General and Scalable Parallelization for ML Computation Graphs
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
GPipe: Efficient training of giant neural networks; Automatically discovery of deep learning architectures using meta learning.