Multimedia technologies for landmark-based vehicle navigation

Most existing Global Positioning System (GPS)-based vehicle navigation systems (also termed route guidance systems) utilize distance within their turn-by-turn navigation directions. For example, a system might give a voice instruction like "turn left in 0.2 mile". However, human drivers usually use landmarks to help their navigation. Some previous research has shown that there are performance-related benefits in using landmarks instead of distance for navigation. The goal of this dissertation is to develop multimedia techniques that can be used for achieving landmark-based navigation using computer vision and machine learning techniques. This dissertation makes contributions in landmark labeling, detection, recognition and the human vehicle interfaces. Landmark labeling is essential for development of landmark recognition systems and using a landmark-based navigation system. The first contribution of this dissertation is a semi-supervised learning-based approach for labeling landmarks in images. The proposed approaches, SmartLabel and SmartLabel-2, minimize user input in labeling landmarks. Text on road signs carries much useful information for driving. The second contribution is an automatic system which detects text on road signs from driving videos. The third contribution is a novel approach for recognizing a given street landmark such as a store sign in a sequence of images. We develop a street landmark recognition system that combines salient region detection, segmentation, and object fingerprint extraction techniques. The fourth contribution is a landmark building recognition framework which is able to recognize and localize the target building in dynamic driving data. Navigation user interfaces have changed dramatically over the last decade due to available electronic maps and GPS devices. However, current navigation systems without landmarks have not fully achieved user satisfaction. The fifth contribution of this dissertation is to demonstrate the concept of landmark-based vehicle navigation on a computer display and also show a prototype using a full-windshield head-up display system.