Zero-Shot Image Feature Consensus with Deep Functional Maps
arxiv(2024)
摘要
Correspondences emerge from large-scale vision models trained for generative
and discriminative tasks. This has been revealed and benchmarked by computing
correspondence maps between pairs of images, using nearest neighbors on the
feature grids. Existing work has attempted to improve the quality of these
correspondence maps by carefully mixing features from different sources, such
as by combining the features of different layers or networks. We point out that
a better correspondence strategy is available, which directly imposes structure
on the correspondence field: the functional map. Wielding this simple
mathematical tool, we lift the correspondence problem from the pixel space to
the function space and directly optimize for mappings that are globally
coherent. We demonstrate that our technique yields correspondences that are not
only smoother but also more accurate, with the possibility of better reflecting
the knowledge embedded in the large-scale vision models that we are studying.
Our approach sets a new state-of-the-art on various dense correspondence tasks.
We also demonstrate our effectiveness in keypoint correspondence and affordance
map transfer.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要