Multimodal Gaze-Based Interaction in Cars: Are Mid-Air Gestures with Haptic Feedback Safer Than Buttons?

We studied two interaction techniques to perform secondary tasks in a driving simulator environment with the focus on driving safety. In both techniques, the participants ( N = 20) used gaze pointing to select virtual task buttons. Toggling the controls was achieved by either mid-air gestures with haptic feedback or physical buttons located on the steering wheel. To evaluate each technique, we compared several measures, such as mean task times, pedestrian detections, lane deviations, and task complexity ratings. The results showed that both techniques allowed operation without severely compromising driving safety. However, interaction using gestures was rated as more complex, caused more fatigue and frustration, and pedestrians were noticed with longer delays than using physical buttons. The results suggest that gaze pointing accuracy was not always sufficient, while mid-air gestures require more robust algorithms before they can offer functionality comparable to interaction with physical buttons.