MagStroke: A Magnetic Based Virtual Keyboard for Off-the-Shelf Smart Devices

We present MagStroke, a system that provides a more flexible mechanism for providing textual input to smart devices with small screens. The key idea is that MagStroke leverages the effect of a magnet worn on the user's hand on a nearby device's magnetometer to enable the use of larger virtualized keyboards. While typing a certain key using a virtual standard keyboard, the hand and fingers of the user, with the attached magnet, move in a unique motion and thus a unique pattern is formed in the device's embedded magnetometer sensor readings. To realize MagStroke, we address several challenges including handling background magnetic noise, keystroke detection and pattern segmentation, feature selection, and training overhead reduction. We implement a proof-of-concept prototype using off-the-shelf mobile devices and extensively evaluate the different modules in our system. Our results show that MagStroke can achieve a word recognition accuracy of 95% with a 46% reduction in training overhead, highlighting its promise as a ubiquitous off-the-shelf magnetic-based keyboard.