A Robotic Skill Learning System Built Upon Diffusion Policies and Foundation Models
arxiv(2024)
摘要
In this paper, we build upon two major recent developments in the field,
Diffusion Policies for visuomotor manipulation and large pre-trained multimodal
foundational models to obtain a robotic skill learning system. The system can
obtain new skills via the behavioral cloning approach of visuomotor diffusion
policies given teleoperated demonstrations. Foundational models are being used
to perform skill selection given the user's prompt in natural language. Before
executing a skill the foundational model performs a precondition check given an
observation of the workspace. We compare the performance of different
foundational models to this end as well as give a detailed experimental
evaluation of the skills taught by the user in simulation and the real world.
Finally, we showcase the combined system on a challenging food serving scenario
in the real world. Videos of all experimental executions, as well as the
process of teaching new skills in simulation and the real world, are available
on the project's website.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要