ReCoRe: Regularized Contrastive Representation Learning of World Model
CoRR(2023)
摘要
While recent model-free Reinforcement Learning (RL) methods have demonstrated
human-level effectiveness in gaming environments, their success in everyday
tasks like visual navigation has been limited, particularly under significant
appearance variations. This limitation arises from (i) poor sample efficiency
and (ii) over-fitting to training scenarios. To address these challenges, we
present a world model that learns invariant features using (i) contrastive
unsupervised learning and (ii) an intervention-invariant regularizer. Learning
an explicit representation of the world dynamics i.e. a world model, improves
sample efficiency while contrastive learning implicitly enforces learning of
invariant features, which improves generalization. However, the naive
integration of contrastive loss to world models fails due to a lack of
supervisory signals to the visual encoder, as world-model-based RL methods
independently optimize representation learning and agent policy. To overcome
this issue, we propose an intervention-invariant regularizer in the form of an
auxiliary task such as depth prediction, image denoising, etc., that explicitly
enforces invariance to style-interventions. Our method outperforms current
state-of-the-art model-based and model-free RL methods and significantly on
out-of-distribution point navigation task evaluated on the iGibson benchmark.
We further demonstrate that our approach, with only visual observations,
outperforms recent language-guided foundation models for point navigation,
which is essential for deployment on robots with limited computation
capabilities. Finally, we demonstrate that our proposed model excels at the
sim-to-real transfer of its perception module on Gibson benchmark.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要