How do Gestures Matter for Mutual Awareness in Cockpits?: Disclosing Interactions through Graphical Representations

This paper presents an approach ensuring mutual awareness among airliner pilots using touch-based interactions. Indeed, touchscreens are making their way into cockpits, but touchbased gestures are less performative than gestures on physical controls, and they are limited in aircraft for efficiency and safety reasons. To support a safer perception by the other pilot, we propose to supplement the perception of performed gestures with graphical representations. Our hypothesis is that representing the effect of gestures is more relevant than representing gestures themselves. We introduce our design choices to build the representations for mutual awareness based on an analysis of the activity and graphical semiology. We report results gathered from walkthroughs of the designs with airliner pilots. These results confirm that representing the effects of gestures is an efficient means for mutual awareness. Our work shows how pilots understand the effect of gesture both as a result and as an impression.