Expert Proximity as Surrogate Rewards for Single Demonstration Imitation Learning
CoRR(2024)
摘要
In this paper, we focus on single-demonstration imitation learning (IL), a
practical approach for real-world applications where obtaining numerous expert
demonstrations is costly or infeasible. In contrast to typical IL settings with
multiple demonstrations, single-demonstration IL involves an agent having
access to only one expert trajectory. We highlight the issue of sparse reward
signals in this setting and propose to mitigate this issue through our proposed
Transition Discriminator-based IL (TDIL) method. TDIL is an IRL method designed
to address reward sparsity by introducing a denser surrogate reward function
that considers environmental dynamics. This surrogate reward function
encourages the agent to navigate towards states that are proximal to expert
states. In practice, TDIL trains a transition discriminator to differentiate
between valid and non-valid transitions in a given environment to compute the
surrogate rewards. The experiments demonstrate that TDIL outperforms existing
IL approaches and achieves expert-level performance in the single-demonstration
IL setting across five widely adopted MuJoCo benchmarks as well as the "Adroit
Door" environment.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要