Voting-Based Backchannel Timing Prediction Using Audio-Visual Information.

While many spoken dialog systems are recently developed, users need to summarize and convey what they want the system to do clearly. However, in a human dialog, a speaker often summarize what to say incrementally, provided that there is a good listener who responds to the speakeru0027s utterances at appropriate timing. We consider that generating backchannel responses, where appropriate, overlapped with the useru0027s utterances is crucial for an artificial listener system that can promote useru0027s utterances since such overlaps are the norm in human dialogs. Toward the goal to realize such a listener system, in this paper, we propose a voting-based algorithm of predicting the end of utterances early (i.e., before the utterances end) using audio-visual information. In the evaluation, we demonstrate the effectiveness of using audio-visual information and the applicability of the voting-based prediction algorithm with some early results.