ActionDiffusion: An Action-aware Diffusion Model for Procedure Planning in Instructional Videos
arxiv(2024)
摘要
We present ActionDiffusion – a novel diffusion model for procedure planning
in instructional videos that is the first to take temporal inter-dependencies
between actions into account in a diffusion model for procedure planning. This
approach is in stark contrast to existing methods that fail to exploit the rich
information content available in the particular order in which actions are
performed. Our method unifies the learning of temporal dependencies between
actions and denoising of the action plan in the diffusion process by projecting
the action information into the noise space. This is achieved 1) by adding
action embeddings in the noise masks in the noise-adding phase and 2) by
introducing an attention mechanism in the noise prediction network to learn the
correlations between different action steps. We report extensive experiments on
three instructional video benchmark datasets (CrossTask, Coin, and NIV) and
show that our method outperforms previous state-of-the-art methods on all
metrics on CrossTask and NIV and all metrics except accuracy on Coin dataset.
We show that by adding action embeddings into the noise mask the diffusion
model can better learn action temporal dependencies and increase the
performances on procedure planning.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要