On Sparsifying Encoder Outputs in Sequence-to-Sequence Models

Zhang Biao
Zhang Biao
Cited by: 0|Bibtex|Views44|Links

Abstract:

Sequence-to-sequence models usually transfer all encoder outputs to the decoder for generation. In this work, by contrast, we hypothesize that these encoder outputs can be compressed to shorten the sequence delivered for decoding. We take Transformer as the testbed and introduce a layer of stochastic gates in-between the encoder and the...More

Code:

Data:

Your rating :
0

 

Tags
Comments