Learning Exhaustive Correlation for Spectral Super-Resolution: Where Spatial-Spectral Attention Meets Linear Dependence
arxiv(2023)
摘要
Spectral super-resolution that aims to recover hyperspectral image (HSI) from
easily obtainable RGB image has drawn increasing interest in the field of
computational photography. The crucial aspect of spectral super-resolution lies
in exploiting the correlation within HSIs. However, two types of bottlenecks in
existing Transformers limit performance improvement and practical applications.
First, existing Transformers often separately emphasize either spatial-wise or
spectral-wise correlation, disrupting the 3D features of HSI and hindering the
exploitation of unified spatial-spectral correlation. Second, existing
self-attention mechanism always establishes full-rank correlation matrix by
learning the correlation between pairs of tokens, leading to its inability to
describe linear dependence widely existing in HSI among multiple tokens. To
address these issues, we propose a novel Exhaustive Correlation Transformer
(ECT) for spectral super-resolution. First, we propose a Spectral-wise
Discontinuous 3D (SD3D) splitting strategy, which models unified
spatial-spectral correlation by integrating spatial-wise continuous splitting
strategy and spectral-wise discontinuous splitting strategy. Second, we propose
a Dynamic Low-Rank Mapping (DLRM) model, which captures linear dependence among
multiple tokens through a dynamically calculated low-rank dependence map. By
integrating unified spatial-spectral attention and linear dependence, our ECT
can model exhaustive correlation within HSI. The experimental results on both
simulated and real data indicate that our method achieves state-of-the-art
performance. Codes and pretrained models will be available later.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要