Real World Offline Reinforcement Learning with Realistic Data Source

arxiv(2022)

引用 11|浏览59
暂无评分
摘要
Offline reinforcement learning (ORL) holds great promise for robot learning due to its ability to learn from arbitrary pre-generated experience. However, current ORL benchmarks are almost entirely in simulation and utilize contrived datasets like replay buffers of online RL agents or sub-optimal trajectories, and thus hold limited relevance for real-world robotics. In this work (Real-ORL), we posit that data collected from safe operations of closely related tasks are more practical data sources for real-world robot learning. Under these settings, we perform an extensive (6500+ trajectories collected over 800+ robot hours and 270+ human labor hour) empirical study evaluating generalization and transfer capabilities of representative ORL methods on four real-world tabletop manipulation tasks. Our study finds that ORL and imitation learning prefer different action spaces, and that ORL algorithms can generalize from leveraging offline heterogeneous data sources and outperform imitation learning. We release our dataset and implementations at URL: https://sites.google.com/view/real-orl
更多
查看译文
关键词
270+ human labor hour,800+ robot hours,arbitrary pre-generated experience,closely related tasks,contrived datasets,current ORL benchmarks,leveraging offline heterogeneous data sources,online RL agents,ORL algorithms,outperform imitation learning,practical data sources,real-world robot learning,real-world robotics,real-world tabletop manipulation tasks,realistic data source,replay buffers,representative ORL methods,safe operations,sub-optimal trajectories,world offline reinforcement learning
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要