Socially-Sensitive Interfaces: From Offline Studies To Interactive Experiences

Recent years have initiated a paradigm shift from pure task-based human-machine interfaces towards socially-sensitive interaction. In addition to what users explicitly say or gesture at, socially-sensitive interfaces are able to sense more subtle human cues, such as head postures and movements, to infer psychological user states, such as attention and affect, and also to enrich system responses with social signals. However, most approaches focus on offline analysis of previously recorded data limiting the investigation to prototypical behaviors in laboratory-like settings. In my presentation, I will focus on challenges that arise when integrating social signal processing techniques into interactive systems designed for real-world applications. From a technical perspective, this requires effective tools able to synchronize, process, and analyze relevant signals in online mode. From a user perspective, appropriate strategies need to be defined to respond to social signals at the right moment in time without disturbing the flow of interaction.I will discuss two interaction styles for socially-sensitive interfaces. In the area of information retrieval, the concept of empathic stimulation has been used to optimize the selection and presentation of data. The basic idea is to exploit sensory data on the users' emotional state to provide them with cues that inspire their curiosity during the data exploration task. In the domain of social coaching, the concept of social augmentation has been employed to give people ambient feedback on their behavior while being engaged in a social interaction. The presentation will be illustrated by examples from various national and international projects following these two interaction styles