Frequency Principle of Attention-Based Convolutional Neural Networks for Single Image Super-Resolution

In this paper, new insights into the effects of attention mechanism on single image super-resolution (SR) are provided. A basic structure of attention modules is proposed to analyze the performance of attention mechanism on SR. The attention modules include the channel attention module, spatial attention module, and self-attention module. Investigations on the attention modules mainly from three aspects. Firstly, the basic building block of the attention module is proposed. Secondly, the effects of the channel attention, spatial attention, and self-attention module on the SR performance are analysed separately. Thirdly, an efficient multi-attention module is obtained by experiments and the frequency principle is used to analyse the training behavior of the multi-attention module in frequency domain. Experiment results of the frequency principle analysis suggest that the proposed multi-attention module can improve the SR performance significantly and avoid instability and performance degradation during training. Further, the output feature maps of the multi-attention module are also provided, and it is observed that the multi-attention module can make SR networks to focus on high-frequency textured regions of the input image properly.