Multi-modal image fusion with the hybrid ℓ0ℓ1 layer decomposing and multi-directional filter banks

The key problem of multi-modal image fusion is that the complementary features of the source images are easily lose in the fusion. In this paper, the fusion algorithm is proposed with the hybrid ℓ0ℓ1 layer decomposing and multi-directional filter banks to extract edge, contour, and detail features from images and fuse the complementary features well. First, the hybrid ℓ0ℓ1 layer decomposing is ability to effectively overcome halo artifacts and over-enhancement, when the detail features are eliminated, so the low-frequency and detail features of the source images are separated. Then, the visual salient detection based on ant colony optimisation and local phase coherenceis is introduced to guide the fusion of the base lay images. Then the detail image is decomposed by using multi-directional filter banker, and the different direction detail features are extracted, and the fusion rule multi-directional gradient and principal component analysis are adopted for the detail sub-band images to prevent the loss of detail features. Finally, the final fused image is reconstructed by the inverse transformation of the hybrid ℓ0ℓ1 layer decomposing. Experimental results demonstrate that the values of the spatial frequency, the the standard deviation, the edge strength,the difference seminary index, and the difference structural similarity are increased significantly, so the proposed fusion algorithm can effectively preserve the complementary information between images and improve the quality of fused images.