Clearer Frames, Anytime: Resolving Velocity Ambiguity in Video Frame Interpolation
CoRR(2023)
摘要
Existing video frame interpolation (VFI) methods blindly predict where each
object is at a specific timestep t ("time indexing"), which struggles to
predict precise object movements. Given two images of a baseball, there are
infinitely many possible trajectories: accelerating or decelerating, straight
or curved. This often results in blurry frames as the method averages out these
possibilities. Instead of forcing the network to learn this complicated
time-to-location mapping implicitly together with predicting the frames, we
provide the network with an explicit hint on how far the object has traveled
between start and end frames, a novel approach termed "distance indexing". This
method offers a clearer learning goal for models, reducing the uncertainty tied
to object speeds. We further observed that, even with this extra guidance,
objects can still be blurry especially when they are equally far from both
input frames (i.e., halfway in-between), due to the directional ambiguity in
long-range motion. To solve this, we propose an iterative reference-based
estimation strategy that breaks down a long-range prediction into several
short-range steps. When integrating our plug-and-play strategies into
state-of-the-art learning-based models, they exhibit markedly sharper outputs
and superior perceptual quality in arbitrary time interpolations, using a
uniform distance indexing map in the same format as time indexing.
Additionally, distance indexing can be specified pixel-wise, which enables
temporal manipulation of each object independently, offering a novel tool for
video editing tasks like re-timing.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要