EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real World
arxiv(2024)
摘要
Being able to map the activities of others into one's own point of view is
one fundamental human skill even from a very early age. Taking a step toward
understanding this human ability, we introduce EgoExoLearn, a large-scale
dataset that emulates the human demonstration following process, in which
individuals record egocentric videos as they execute tasks guided by
demonstration videos. Focusing on the potential applications in daily
assistance and professional support, EgoExoLearn contains egocentric and
demonstration video data spanning 120 hours captured in daily life scenarios
and specialized laboratories. Along with the videos we record high-quality gaze
data and provide detailed multimodal annotations, formulating a playground for
modeling the human ability to bridge asynchronous procedural actions from
different viewpoints. To this end, we present benchmarks such as cross-view
association, cross-view action planning, and cross-view referenced skill
assessment, along with detailed analysis. We expect EgoExoLearn can serve as an
important resource for bridging the actions across views, thus paving the way
for creating AI agents capable of seamlessly learning by observing humans in
the real world. Code and data can be found at:
https://github.com/OpenGVLab/EgoExoLearn
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要