Fast and Simple Mixture of Softmaxes with BPE and Hybrid-LightRNN for Language Generation
national conference on artificial intelligence, 2019.
Mixture of Softmaxes (MoS) has been shown to be effective at addressing the expressiveness limitation of Softmax-based models. Despite the known advantage, MoS is practically sealed by its large consumption of memory and computational time due to the need of computing multiple Softmaxes. In this work, we set out to unleash the power of Mo...More
PPT (Upload PPT)