Automatic registration of optical imagery with 3D LiDAR data using statistical similarity

The development of robust and accurate methods for automatic registration of optical imagery and 3D LiDAR data continues to be a challenge for a variety of applications in photogrammetry, computer vision and remote sensing. This paper proposes a new approach for the registration of optical imagery with LiDAR data based on the theory of Mutual Information (MI), which exploits the statistical dependency between same- and multi-modal datasets to achieve accurate registration. The MI-based similarity measures quantify dependencies between aerial imagery, and both LiDAR intensity data and 3D point cloud data. The needs for specific physical feature correspondences, which are not always attainable in the registration of imagery with 3D point clouds, are avoided. Current methods for registering 2D imagery to 3D point clouds are first reviewed, after which the mutual MI approach is presented. Particular attention is given to adoption of the Normalised Combined Mutual Information (NCMI) approach as a means to produce a similarity measure that exploits the inherently registered LiDAR intensity and point cloud data so as to improve the robustness of registration between optical imagery and LiDAR data. The effectiveness of local versus global similarity measures is also investigated, as are the transformation models involved in the registration process. An experimental program conducted to evaluate MI-based methods for registering aerial imagery to LiDAR data is reported and the results obtained in two areas with differing terrain and land cover, and with aerial imagery of different resolution and LiDAR data with different point density are discussed. These results demonstrate the potential of the MI and especially the CMI methods for registration of imagery and 3D point clouds, and they highlight the feasibility and robustness of the presented MI-based approach to automated registration of multi-sensor, multi-temporal and multi-resolution remote sensing data for a wide range of applications.