Analyzing Feed-Forward Blocks in Transformers through the Lens of Attention Maps
arxiv(2023)
摘要
Transformers are ubiquitous in wide tasks. Interpreting their internals is a
pivotal goal. Nevertheless, their particular components, feed-forward (FF)
blocks, have typically been less analyzed despite their substantial parameter
amounts. We analyze the input contextualization effects of FF blocks by
rendering them in the attention maps as a human-friendly visualization scheme.
Our experiments with both masked- and causal-language models reveal that FF
networks modify the input contextualization to emphasize specific types of
linguistic compositions. In addition, FF and its surrounding components tend to
cancel out each other's effects, suggesting potential redundancy in the
processing of the Transformer layer.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要