Is anyone looking?: mediating shoulder surfing on public displays (the video).

When a person interacts with a display in an open area, sensitive information becomes visible to shoulder-surfing passers-by. While a person's body shields small displays, shielding is less effective as display area increases. To mitigate this problem, we sense spatial relationships between the passerby, person and display. Awareness of onlookers is provided through visual cues: flashing screen borders, a 3D model mirroring the onlooker's position and gaze, and an indicator that illustrates their gaze direction. The person can react with a gesture that commands the display to black out personal windows, or to collect them on one side. Alternately, the display will automatically darken screen regions visible by the onlooker, but leaving the display area shielded by the person's body unaltered (thus allowing the person to continue their actions). The person can also invite the onlooker to collaborate with them via a gesture that reverses these protective mechanisms. This video illustrates these and other approaches to mitigate shoulder surfing.