Multimodal gait analysis based on wearable inertial and microphone sensors

With the quick development of smart tiny sensors, gait analysis (GA) methods based on wearable devices have become more popular recently. However, most existing wearable GA methods focus on data analysis from inertial sensors. This paper firstly proposes a novel system based on low-cost, wearable and multimodal sensors for GA, which combines inertial sensor and microphone sensor. The system is introduced by including the wearable multimodal prototype device, related software program, data collection procedure, etc. Secondly, based on the system, a freely available database (ICT_Gait) is published by including the details of setup method and spatial-temporal data label method. This database provides both data from inertial sensor and microphone sensor on two feet from 15 healthy subjects. There are totally 240 data sequences and 1732 steps from three various collection strategies. And this is the first database for multimodal and wearable GA as we know. Finally, based on the system and database, a preliminary research is conducted for footstep detection - the basic and key process for GA. And a coarse-to-fine multimodal fusion algorithm is presented for footstep detection. Experimental results show it achieves average 99.72% for precision rate and 97.83% for recall rate using multimodal fusion algorithm, which presents the effectiveness of multimodal fusion for wearable GA.