Appliance Control by TDoA-based Localization and Gesture Recognition

Smart home is one of the major Internet of Things (IoT) applications. When realizing a smart home, designing an intuitive interaction for appliance control has become a crucial issue. A control intention to an appliance comprises two main meanings, identification and control command. But, previous works seldom deliver these two meanings at the same time. To fully convey the control intention, this paper proposes a scheme of intuitive appliance control that tracks and recognizes hand moving trajectory and hand gestures respectively by exploiting acoustic information. We utilize a 3D positioning method based on time difference of arrival (TDoA) of acoustic signals to detect the trajectory of a hand movement for appliance identification and design a gesture recognition method by Doppler effect for control command. We validate the effectiveness of the proposed scheme via real-life experiments. The results show that average locating errors to the targets are 5.24° and 10.35° in the horizontal and vertical planes respectively and the accuracy of gesture recognition is 96.66%. Since human's eyesight is horizontal to the ground, locating an object in the vertical view is error-prone. We also provide a scroll gesture to assist the appliance selection.