Distilling Conditional Diffusion Models for Offline Reinforcement Learning through Trajectory Stitching
CoRR(2024)
摘要
Deep generative models have recently emerged as an effective approach to
offline reinforcement learning. However, their large model size poses
challenges in computation. We address this issue by proposing a knowledge
distillation method based on data augmentation. In particular, high-return
trajectories are generated from a conditional diffusion model, and they are
blended with the original trajectories through a novel stitching algorithm that
leverages a new reward generator. Applying the resulting dataset to behavioral
cloning, the learned shallow policy whose size is much smaller outperforms or
nearly matches deep generative planners on several D4RL benchmarks.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要