DECIDER: A Rule-Controllable Decoding Strategy for Language Generation by Imitating Dual-System Cognitive Theory
arxiv(2024)
摘要
Lexicon-based constrained decoding approaches aim to control the meaning or
style of the generated text through certain target concepts. Existing
approaches over-focus the targets themselves, leading to a lack of high-level
reasoning about how to achieve them. However, human usually tackles tasks by
following certain rules that not only focuses on the targets but also on
semantically relevant concepts that induce the occurrence of targets. In this
work, we present DECIDER, a rule-controllable decoding strategy for constrained
language generation inspired by dual-system cognitive theory. Specifically, in
DECIDER, a pre-trained language model (PLM) is equiped with a logic reasoner
that takes high-level rules as input. Then, the DECIDER allows rule signals to
flow into the PLM at each decoding step. Extensive experimental results
demonstrate that DECIDER can effectively follow given rules to guide generation
direction toward the targets in a more human-like manner.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要