Heterogeneous Information Fusion and Visualization for a Large-Scale Intelligent Video Surveillance System

Wide-area monitoring for a smart community can be challenging in systems engineering because of its large scale and heterogeneity at the sensor, algorithm, and visualization levels. A smart interface to visualize high-level information fused from a diversity of low-level surveillance data, and to facilitate rapid response of events, is critical for the design of the system. This paper presents an event-driven visualization mechanism fusing multimodal information for a large-scale intelligent video surveillance system. The mechanism proactively helps security personnel intuitively be aware of events through close cooperation among visualization, data fusion, and sensor tasking. The visualization not only displays 2-D, 3-D, and geographical information within a condensed form of interface but also automatically shows the only important video streams corresponding to spontaneous alerts and events by a decision process called display switching arbitration. The display switching arbitration decides the importance of cameras by score ranking that considers event urgency and semantic object features. This system has been successfully deployed in a campus to demonstrate its usability and efficiency for an installation with two camera clusters that include dozens of cameras, and with a lot of video analytics to detect alerts and events. A further simulation comparing the display switching arbitration with similar camera selection methods shows that our method improves the visualization by selecting better representative camera views and reducing redundant switchover among multiview videos.