A registration strategy from scale adjustment to fine for thermal-visible face images

•A two-phase strategy for thermal and visible face image registration is proposed.•To adjust the large-scale differences in thermal-visible face image pair, a calibration rig-based pre-transformation is implemented. Further, fine registration is applied using phase congruency to precise the alignment.•A particle swarm optimization based fast sample consensus strategy is introduced to select inlier correspondence.•Experimental results show the effectiveness of the proposed method in comparison of the state-of-the-art methods.