Fusion of Multiple Images by Higher-Order SVD of Third-Order Image Tensors

An important topic in image fusion is methodology to combine multiple digital images for visual display or for processing such as traditional edge detection or data mining to find imagerelated structure. This paper describes a method of fusion for applications in which (1) input images have been registered, sized, and scaled in pixel intensities suitably for mutual comparison (for example, inputs from a multi-lens array), and (2) information about edges and lines is desired. The fusion is a sequence of computations. The first step organizes the input images as a third-order tensor A and computes a higher-order generalization of singular value decomposition (abbreviated HOSVD) for A. HOSVD creates a subtensor B containing a set of images that are linear fusions of the inputs, orthogonal, and ordered by decreasing norm. The second step is phase analysis of the basis images in B which extracts edge-line information by computing image phase maps. The third step fuses the raw phase maps themselves by local energy criteria, i.e., by pixel-wise square root of the sum of the squares. Finally, the fused maps are combined with input images for visualization and can be used in other processing, for instance, to delineate regions by connected line segments. In some applications additional input images are acquired after the initial HOSVD, in which case incremental HOSVD is an effective way to update an existing decomposition without recomputing the entire HOSVD from scratch. Reduced dimension HOSVD is an option for reducing the storage requirements but introduces an error that must be acceptable in an application. Examples of fusion are given using multimodal images and multiresolution images.