Reliable Multiscale and Multiwindow Stereo Matching

We consider the two-images stereo disparity problem favoring correctness of matches over density. We will deal with high resolution images which permit an accurate matching in textured zones, but which might present, as any stereo pair, ambiguities and occlusions. Global variational methods can estimate a dense map based on regularity assumptions about the disparity function. However, if these assumptions are incorrect this may lead to erroneous interpretations and mismatches. The availability of tristereo or even multiview stereo imagery permit us to combine the disparities from different pairs, allowing for a reliable densification not based on regularity assumptions. Local methods are suitable for this purpose since they permit us to check the validity of each match. The main disadvantages of local methods are the matching ambiguity and the failures of the fronto-parallel hypothesis (at places like discontinuities and slanted surfaces). We advocate, in this work, for the use of oriented windows in order to deal with slanted surfaces and discontinuities. Unlike adaptive support windows the oriented windows permit us to correctly estimate disparities on non-fronto-parallel surfaces. Several parameterless techniques for detecting mismatches are presented. The incorporation of these validation techniques in a coarse-to-fine multiwindow algorithm, allows us to obtain fairly dense results with few mismatches. An extensive comparison, including classical stereo pairs, high resolution satellite images, and images from the KITTI benchmark, illustrates the performance of the proposed method.