ALERT-Transformer: Bridging Asynchronous and Synchronous Machine Learning for Real-Time Event-based Spatio-Temporal Data
CoRR(2024)
摘要
We seek to enable classic processing of continuous ultra-sparse
spatiotemporal data generated by event-based sensors with dense machine
learning models. We propose a novel hybrid pipeline composed of asynchronous
sensing and synchronous processing that combines several ideas: (1) an
embedding based on PointNet models – the ALERT module – that can continuously
integrate new and dismiss old events thanks to a leakage mechanism, (2) a
flexible readout of the embedded data that allows to feed any downstream model
with always up-to-date features at any sampling rate, (3) exploiting the input
sparsity in a patch-based approach inspired by Vision Transformer to optimize
the efficiency of the method. These embeddings are then processed by a
transformer model trained for object and gesture recognition. Using this
approach, we achieve performances at the state-of-the-art with a lower latency
than competitors. We also demonstrate that our asynchronous model can operate
at any desired sampling rate.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要