Lightweight feature point detection network with channel enhancement

Learning-based local feature point extraction work has made great research progress in related computer vision tasks. However, it is difficult to balance the detection speed and detection efficiency of general deep neural networks, which are affected by illumination and viewpoint changes in outdoor environments. To this end, a lightweight image feature point efficient detection network jointly trained by detector–descriptor is proposed. The shared encoding of the feature map is carried out through the multi-branch structure, and a shuffle block model is designed to enhance the channel features. The enhanced feature is divided into three branches to further encode the feature map. One branch improves the network’s ability to identify feature information through a lightweight attention mechanism, and the other two branches perform feature aggregation on the detector and the descriptor through skip connection layers to enrich image features. Considering the problem of network operation efficiency, the multi-branch coding structure is coupled into a set of parameters corresponding to the plane forward propagation structure through the structure re-parameterization technology, which can reduce the complexity of the network and improve the inference speed of the network. In addition, the correct convergence of the network is guaranteed by constraints on the classification loss and position loss, and the sparsely sampled descriptors are used for training to reduce the computational cost. Experimental results on the Hpatches and KITTI datasets show that the proposed method reduces the parameters and GFLOPs by 10.7% and 61.6%, respectively, compared with recent lightweight work, while maintaining comparable accuracy. Moreover, the proposed method achieves a detection speed of 55FPS on images with a resolution of 480×640. https://github.com/SpiritAshes/Channel-Enhancement.