Lightweight Cross-Modal Information Measure and Propagation for Road Extraction from Remote Sensing Image and Trajectory/LiDAR

Recent studies have confirmed that GPS trajectory can effectively assist in achieving more accurate road extraction from remote sensing images. Therefore, lots of efforts focus on designing effective multi-modal fusion strategies for GPS trajectory and remote sensing image modalities. However, there are still some limitations, e.g ., the fusion structures are complex and hinder further improvements. Moreover, the negative impact of redundant information in various modalities is commonly ignored. This paper aims to design a simple yet effective fusion strategy for GPS trajectory and remote sensing image modalities to address the above issues. Inspired by the network pruning algorithm, we design a Cross-Modal Information Propagation (CMIP) mechanism. CMIP utilizes the scaling factors and sparse constraint to distinguish the redundant information of a certain modality that is directly replaced with corresponding information of another modality. We improve the widely-used L1 sparse constraint and propose a novel information balanced constraint which is added on the scaling factors to better identify and prune redundant channels. Embedded in the CMIP mechanism, a multi-modal information propagation network (CMIPNet) is proposed, which can fully explore the complementarities between different modalities to accurately locate roads, especially roads with noise or incomplete information of a certain modality. Since the CMIP is parameter-free and self-adaptive, CMIPNet is lightweight and easy to deploy. The parameter number of CMIPNet can be comparable to single-modal models, which is about 1/3 of the existing multi-modal models. Extensive experiments are performed to demonstrate that CMIPNet outperforms the previous single- and multi-modal road extraction methods.