Analyzing Narrative Processing in Large Language Models (LLMs): Using GPT4 to test BERT
arxiv(2024)
摘要
The ability to transmit and receive complex information via language is
unique to humans and is the basis of traditions, culture and versatile social
interactions. Through the disruptive introduction of transformer based large
language models (LLMs) humans are not the only entity to "understand" and
produce language any more. In the present study, we have performed the first
steps to use LLMs as a model to understand fundamental mechanisms of language
processing in neural networks, in order to make predictions and generate
hypotheses on how the human brain does language processing. Thus, we have used
ChatGPT to generate seven different stylistic variations of ten different
narratives (Aesop's fables). We used these stories as input for the open source
LLM BERT and have analyzed the activation patterns of the hidden units of BERT
using multi-dimensional scaling and cluster analysis. We found that the
activation vectors of the hidden units cluster according to stylistic
variations in earlier layers of BERT (1) than narrative content (4-5). Despite
the fact that BERT consists of 12 identical building blocks that are stacked
and trained on large text corpora, the different layers perform different
tasks. This is a very useful model of the human brain, where self-similar
structures, i.e. different areas of the cerebral cortex, can have different
functions and are therefore well suited to processing language in a very
efficient way. The proposed approach has the potential to open the black box of
LLMs on the one hand, and might be a further step to unravel the neural
processes underlying human language processing and cognition in general.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要