Event-based Continuous Color Video Decompression from Single Frames
CoRR(2023)
摘要
We present ContinuityCam, a novel approach to generate a continuous video
from a single static RGB image, using an event camera. Conventional cameras
struggle with high-speed motion capture due to bandwidth and dynamic range
limitations. Event cameras are ideal sensors to solve this problem because they
encode compressed change information at high temporal resolution. In this work,
we propose a novel task called event-based continuous color video
decompression, pairing single static color frames and events to reconstruct
temporally continuous videos. Our approach combines continuous long-range
motion modeling with a feature-plane-based synthesis neural integration model,
enabling frame prediction at arbitrary times within the events. Our method does
not rely on additional frames except for the initial image, increasing, thus,
the robustness to sudden light changes, minimizing the prediction latency, and
decreasing the bandwidth requirement. We introduce a novel single objective
beamsplitter setup that acquires aligned images and events and a novel and
challenging Event Extreme Decompression Dataset (E2D2) that tests the method in
various lighting and motion profiles. We thoroughly evaluate our method through
benchmarking reconstruction as well as various downstream tasks. Our approach
significantly outperforms the event- and image- based baselines in the proposed
task.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要