PREGO: online mistake detection in PRocedural EGOcentric videos
CVPR 2024(2024)
摘要
Promptly identifying procedural errors from egocentric videos in an online
setting is highly challenging and valuable for detecting mistakes as soon as
they happen. This capability has a wide range of applications across various
fields, such as manufacturing and healthcare. The nature of procedural mistakes
is open-set since novel types of failures might occur, which calls for
one-class classifiers trained on correctly executed procedures. However, no
technique can currently detect open-set procedural mistakes online. We propose
PREGO, the first online one-class classification model for mistake detection in
PRocedural EGOcentric videos. PREGO is based on an online action recognition
component to model the current action, and a symbolic reasoning module to
predict the next actions. Mistake detection is performed by comparing the
recognized current action with the expected future one. We evaluate PREGO on
two procedural egocentric video datasets, Assembly101 and Epic-tent, which we
adapt for online benchmarking of procedural mistake detection to establish
suitable benchmarks, thus defining the Assembly101-O and Epic-tent-O datasets,
respectively.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要