Symmetry Considerations for Learning Task Symmetric Robot Policies
arxiv(2024)
摘要
Symmetry is a fundamental aspect of many real-world robotic tasks. However,
current deep reinforcement learning (DRL) approaches can seldom harness and
exploit symmetry effectively. Often, the learned behaviors fail to achieve the
desired transformation invariances and suffer from motion artifacts. For
instance, a quadruped may exhibit different gaits when commanded to move
forward or backward, even though it is symmetrical about its torso. This issue
becomes further pronounced in high-dimensional or complex environments, where
DRL methods are prone to local optima and fail to explore regions of the state
space equally. Past methods on encouraging symmetry for robotic tasks have
studied this topic mainly in a single-task setting, where symmetry usually
refers to symmetry in the motion, such as the gait patterns. In this paper, we
revisit this topic for goal-conditioned tasks in robotics, where symmetry lies
mainly in task execution and not necessarily in the learned motions themselves.
In particular, we investigate two approaches to incorporate symmetry invariance
into DRL – data augmentation and mirror loss function. We provide a
theoretical foundation for using augmented samples in an on-policy setting.
Based on this, we show that the corresponding approach achieves faster
convergence and improves the learned behaviors in various challenging robotic
tasks, from climbing boxes with a quadruped to dexterous manipulation.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要