DiffStitch: Boosting Offline Reinforcement Learning with Diffusion-based Trajectory Stitching
CoRR(2024)
摘要
In offline reinforcement learning (RL), the performance of the learned policy
highly depends on the quality of offline datasets. However, in many cases, the
offline dataset contains very limited optimal trajectories, which poses a
challenge for offline RL algorithms as agents must acquire the ability to
transit to high-reward regions. To address this issue, we introduce
Diffusion-based Trajectory Stitching (DiffStitch), a novel diffusion-based data
augmentation pipeline that systematically generates stitching transitions
between trajectories. DiffStitch effectively connects low-reward trajectories
with high-reward trajectories, forming globally optimal trajectories to address
the challenges faced by offline RL algorithms. Empirical experiments conducted
on D4RL datasets demonstrate the effectiveness of DiffStitch across RL
methodologies. Notably, DiffStitch demonstrates substantial enhancements in the
performance of one-step methods (IQL), imitation learning methods (TD3+BC), and
trajectory optimization methods (DT).
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要