On the correspondence between objects and events for the diagnosis of situations in visual surveillance tasks

A key problem in visual surveillance systems (VSS) is to find an effective procedure for linking the geometric descriptions of a scene at the object level with the corresponding descriptions of the agents intervening in this scene at the activity level. In this work, we explore a constructivist approach based on using the usual Artificial Intelligence (AI) techniques and methods to establish correspondences between the entities and relations of the ontologies in these two levels. The proposal is exemplified using a real interior scenario that uses images from just one fixed camera and where the purpose of the surveillance is to do a preventive diagnosis of the activity of abandoning a potentially dangerous object in a sensitive area. The work stresses: (1) anchoring the object-level labels in the result of analytical processes on blobs, (2) specifying contextual knowledge that has to be injected to link the activities, as described by a human surveillance expert, with the objects, as they are labelled by the same expert from geometric descriptions. The work is set within the context of the 50th anniversary of AI and the leading theories on human visual perception.