Uncovering Latent Human Wellbeing in Language Model Embeddings
CoRR(2024)
摘要
Do language models implicitly learn a concept of human wellbeing? We explore
this through the ETHICS Utilitarianism task, assessing if scaling enhances
pretrained models' representations. Our initial finding reveals that, without
any prompt engineering or finetuning, the leading principal component from
OpenAI's text-embedding-ada-002 achieves 73.9
the 74.6
pretraining conveys some understanding about human wellbeing. Next, we consider
four language model families, observing how Utilitarianism accuracy varies with
increased parameters. We find performance is nondecreasing with increased model
size when using sufficient numbers of principal components.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要