HUBERT Untangles BERT to Improve Transfer across NLP Tasks
arxiv(2019)
摘要
We introduce HUBERT which combines the structured-representational power of Tensor-Product Representations (TPRs) and BERT, a pre-trained bidirectional Transformer language model. We show that there is shared structure between different NLP datasets that HUBERT, but not BERT, is able to learn and leverage. We validate the effectiveness of our model on the GLUE benchmark and HANS dataset. Our experiment results show that untangling data-specific semantics from general language structure is key for better transfer among NLP tasks.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络