Human-In-The-Loop Machine Learning with Intelligent Multimodal Interfaces

In this paper, we argue that intelligent multimodal interfaces add an important dimension for advancing the cause of human-in-the-loop machine learning (HITL-ML). Multimodal interfaces seek to leverage natural human capabilities to communicate via speech, gesture, touch etc. Such interfaces are said to be intelligent when they can better learn and adapt to the requirements and condition of a user. Here we show how this implicit learning of system parameters (e.g. via interaction feedback loop) and labelling of user cognitive states is an effective and often overlooked dimension of HITL-ML. We also present a research brief of relevant investigations undertaken in this regard.