Multi-Touch interactions for control and display in interactive cockpits: issues and a proposal

Over the last few years multi-touch interfaces have made their ways in most environments including mobile technologies, flight entertainment systems, consumer electronics, ... Such interfaces and associated interaction techniques have demonstrated benefits partly due to the fact that the output device integrates input management thus bridging the (classical) gap between input and output in user interfaces. They have also demonstrated benefits in terms of performance for triggering commands by exploiting multi-fingers interactions thus reducing the number of un-necessary modes. Together with these benefits, multi-touch interfaces bring a set of issues that are still to be solved prior to make them acceptable for command and control of (safety) critical interactive systems. Such issues include usability and reliability in a way that is even more salient than in classical WIMP (Windows, Icons, Menus, Pointing device) interfaces that have been around for more than 30 years. This paper proposes a notation and its associated tool for describing in a complete and unambiguous way multi-touch interactions thus mainly targeting at reliability. We present from a case study (in the application domain of interactive cockpits of aircrafts) how the notation makes it possible to describe such interactions from the hardware level (input devices) to the user application level.