Learning with Language-Guided State Abstractions
ICLR 2024(2024)
摘要
We describe a framework for using natural language to design state
abstractions for imitation learning. Generalizable policy learning in
high-dimensional observation spaces is facilitated by well-designed state
representations, which can surface important features of an environment and
hide irrelevant ones. These state representations are typically manually
specified, or derived from other labor-intensive labeling procedures. Our
method, LGA (language-guided abstraction), uses a combination of natural
language supervision and background knowledge from language models (LMs) to
automatically build state representations tailored to unseen tasks. In LGA, a
user first provides a (possibly incomplete) description of a target task in
natural language; next, a pre-trained LM translates this task description into
a state abstraction function that masks out irrelevant features; finally, an
imitation policy is trained using a small number of demonstrations and
LGA-generated abstract states. Experiments on simulated robotic tasks show that
LGA yields state abstractions similar to those designed by humans, but in a
fraction of the time, and that these abstractions improve generalization and
robustness in the presence of spurious correlations and ambiguous
specifications. We illustrate the utility of the learned abstractions on mobile
manipulation tasks with a Spot robot.
更多查看译文
关键词
human-ai interaction,state abstractions
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要