Visual Imitation Learning with Calibrated Contrastive Representation
CoRR(2024)
摘要
Adversarial Imitation Learning (AIL) allows the agent to reproduce expert
behavior with low-dimensional states and actions. However, challenges arise in
handling visual states due to their less distinguishable representation
compared to low-dimensional proprioceptive features. While existing methods
resort to adopt complex network architectures or separate the process of
learning representation and decision-making, they overlook valuable intra-agent
information within demonstrations. To address this problem, this paper proposes
a simple and effective solution by incorporating calibrated contrastive
representative learning into visual AIL framework. Specifically, we present an
image encoder in visual AIL, utilizing a combination of unsupervised and
supervised contrastive learning to extract valuable features from visual
states. Based on the fact that the improved agent often produces demonstrations
of varying quality, we propose to calibrate the contrastive loss by treating
each agent demonstrations as a mixed sample. The incorporation of contrastive
learning can be jointly optimized with the AIL framework, without modifying the
architecture or incurring significant computational costs. Experimental results
on DMControl Suite demonstrate our proposed method is sample efficient and can
outperform other compared methods from different aspects.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要