Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoning
CoRR(2024)
摘要
Large Language Models (LLMs) demonstrate remarkable proficiency in
comprehending and handling text-based tasks. Many efforts are being made to
transfer these attributes to video modality, which are termed Video-LLMs.
However, existing Video-LLMs can only capture the coarse-grained semantics and
are unable to effectively handle tasks related to comprehension or localization
of specific video segments. In light of these challenges, we propose Momentor,
a Video-LLM capable of accomplishing fine-grained temporal understanding tasks.
To support the training of Momentor, we design an automatic data generation
engine to construct Moment-10M, a large-scale video instruction dataset with
segment-level instruction data. We train Momentor on Moment-10M, enabling it to
perform segment-level reasoning and localization. Zero-shot evaluations on
several tasks demonstrate that Momentor excels in fine-grained temporally
grounded comprehension and localization.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要