Real World Offline Reinforcement Learning with Realistic Data Source


引用 11|浏览59
Offline reinforcement learning (ORL) holds great promise for robot learning due to its ability to learn from arbitrary pre-generated experience. However, current ORL benchmarks are almost entirely in simulation and utilize contrived datasets like replay buffers of online RL agents or sub-optimal trajectories, and thus hold limited relevance for real-world robotics. In this work (Real-ORL), we posit that data collected from safe operations of closely related tasks are more practical data sources for real-world robot learning. Under these settings, we perform an extensive (6500+ trajectories collected over 800+ robot hours and 270+ human labor hour) empirical study evaluating generalization and transfer capabilities of representative ORL methods on four real-world tabletop manipulation tasks. Our study finds that ORL and imitation learning prefer different action spaces, and that ORL algorithms can generalize from leveraging offline heterogeneous data sources and outperform imitation learning. We release our dataset and implementations at URL:
270+ human labor hour,800+ robot hours,arbitrary pre-generated experience,closely related tasks,contrived datasets,current ORL benchmarks,leveraging offline heterogeneous data sources,online RL agents,ORL algorithms,outperform imitation learning,practical data sources,real-world robot learning,real-world robotics,real-world tabletop manipulation tasks,realistic data source,replay buffers,representative ORL methods,safe operations,sub-optimal trajectories,world offline reinforcement learning
AI 理解论文
Chat Paper