ExACT: Language-guided Conceptual Reasoning and Uncertainty Estimation for Event-based Action Recognition and More
CVPR 2024(2024)
摘要
Event cameras have recently been shown beneficial for practical vision tasks,
such as action recognition, thanks to their high temporal resolution, power
efficiency, and reduced privacy concerns. However, current research is hindered
by 1) the difficulty in processing events because of their prolonged duration
and dynamic actions with complex and ambiguous semantics and 2) the redundant
action depiction of the event frame representation with fixed stacks. We find
language naturally conveys abundant semantic information, rendering it
stunningly superior in reducing semantic uncertainty. In light of this, we
propose ExACT, a novel approach that, for the first time, tackles event-based
action recognition from a cross-modal conceptualizing perspective. Our ExACT
brings two technical contributions. Firstly, we propose an adaptive
fine-grained event (AFE) representation to adaptively filter out the repeated
events for the stationary objects while preserving dynamic ones. This subtly
enhances the performance of ExACT without extra computational cost. Then, we
propose a conceptual reasoning-based uncertainty estimation module, which
simulates the recognition process to enrich the semantic representation. In
particular, conceptual reasoning builds the temporal relation based on the
action semantics, and uncertainty estimation tackles the semantic uncertainty
of actions based on the distributional representation. Experiments show that
our ExACT achieves superior recognition accuracy of 94.83
90.10
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要