Integrated data and knowledge driven methodology for human activity recognition

Human activity recognition has been a popular research area concerned with identifying the specific movement or action of a person based on variety of sensor data. Conventional human activity recognition approaches are mainly data driven, which are not working well for composite activity recognition due to the complexity and uncertainty of real scenarios. We propose in this paper a hierarchical structure-based framework and methodology for human activity recognition by an integration of data-driven approach and knowledge-based approach, which provides an interesting framework capable of bridging lower-level pattern recognition and higher-level knowledge for reasoning and explanation. More specifically, this approach constructs a hierarchical structure for representing the composite activity by a composition of lower-level actions and gestures according to its semantic meaning. This hierarchical structure is then transformed into formal syntactical logical formulas and rules, based on which the resolution based automated reasoning is applied to recognize the composite activity given the recognized lower-level actions by using data driven machine learning methods. The work is the validated using some open-source data about video based human activity recognition. The present work provides a promising framework and application illustration of integration of machine learning and symbolic reasoning.