Learning Quadruped Locomotion Using Differentiable Simulation
arxiv(2024)
摘要
While most recent advancements in legged robot control have been driven by
model-free reinforcement learning, we explore the potential of differentiable
simulation. Differentiable simulation promises faster convergence and more
stable training by computing low-variant first-order gradients using the robot
model, but so far, its use for legged robot control has remained limited to
simulation. The main challenge with differentiable simulation lies in the
complex optimization landscape of robotic tasks due to discontinuities in
contact-rich environments, e.g., quadruped locomotion. This work proposes a
new, differentiable simulation framework to overcome these challenges. The key
idea involves decoupling the complex whole-body simulation, which may exhibit
discontinuities due to contact, into two separate continuous domains.
Subsequently, we align the robot state resulting from the simplified model with
a more precise, non-differentiable simulator to maintain sufficient simulation
accuracy. Our framework enables learning quadruped walking in minutes using a
single simulated robot without any parallelization. When augmented with GPU
parallelization, our approach allows the quadruped robot to master diverse
locomotion skills, including trot, pace, bound, and gallop, on challenging
terrains in minutes. Additionally, our policy achieves robust locomotion
performance in the real world zero-shot. To the best of our knowledge, this
work represents the first demonstration of using differentiable simulation for
controlling a real quadruped robot. This work provides several important
insights into using differentiable simulations for legged locomotion in the
real world.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要