Novelty Detection in Reinforcement Learning with World Models
arxiv(2023)
摘要
Reinforcement learning (RL) using world models has found significant recent
successes. However, when a sudden change to world mechanics or properties
occurs then agent performance and reliability can dramatically decline. We
refer to the sudden change in visual properties or state transitions as
novelties. Implementing novelty detection within generated world model
frameworks is a crucial task for protecting the agent when deployed. In this
paper, we propose straightforward bounding approaches to incorporate novelty
detection into world model RL agents, by utilizing the misalignment of the
world model's hallucinated states and the true observed states as an anomaly
score. We provide effective approaches to detecting novelties in a distribution
of transitions learned by an agent in a world model. Finally, we show the
advantage of our work in a novel environment compared to traditional machine
learning novelty detection methods as well as currently accepted RL focused
novelty detection algorithms.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要