Offline Diversity Maximization Under Imitation Constraints
arxiv(2023)
摘要
There has been significant recent progress in the area of unsupervised skill
discovery, utilizing various information-theoretic objectives as measures of
diversity. Despite these advances, challenges remain: current methods require
significant online interaction, fail to leverage vast amounts of available
task-agnostic data and typically lack a quantitative measure of skill utility.
We address these challenges by proposing a principled offline algorithm for
unsupervised skill discovery that, in addition to maximizing diversity, ensures
that each learned skill imitates state-only expert demonstrations to a certain
degree. Our main analytical contribution is to connect Fenchel duality,
reinforcement learning, and unsupervised skill discovery to maximize a mutual
information objective subject to KL-divergence state occupancy constraints.
Furthermore, we demonstrate the effectiveness of our method on the standard
offline benchmark D4RL and on a custom offline dataset collected from a 12-DoF
quadruped robot for which the policies trained in simulation transfer well to
the real robotic system.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要