EventEgo3D: 3D Human Motion Capture from Egocentric Event Streams
CVPR 2024(2024)
摘要
Monocular egocentric 3D human motion capture is a challenging and actively
researched problem. Existing methods use synchronously operating visual sensors
(e.g. RGB cameras) and often fail under low lighting and fast motions, which
can be restricting in many applications involving head-mounted devices. In
response to the existing limitations, this paper 1) introduces a new problem,
i.e., 3D human motion capture from an egocentric monocular event camera with a
fisheye lens, and 2) proposes the first approach to it called EventEgo3D
(EE3D). Event streams have high temporal resolution and provide reliable cues
for 3D human motion capture under high-speed human motions and rapidly changing
illumination. The proposed EE3D framework is specifically tailored for learning
with event streams in the LNES representation, enabling high 3D reconstruction
accuracy. We also design a prototype of a mobile head-mounted device with an
event camera and record a real dataset with event observations and the
ground-truth 3D human poses (in addition to the synthetic dataset). Our EE3D
demonstrates robustness and superior 3D accuracy compared to existing solutions
across various challenging experiments while supporting real-time 3D pose
update rates of 140Hz.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要