The Use of Harr-like Features in Bubblegrams: a Mixed Reality Human-Robot Interaction Technique

We present the application of a vision algorithm based on Harr-like features in Bubble- grams - a new mixed reality-based human-robot in- teraction (HRI) technique. Bubblegrams allows hu- mans and robots working on collocated synchronous tasks to interact directly by visually augmenting their shared physical environment. Bubblegrams uses comics-like interactive graphic balloons or bub- bles that appear above the robot's body and allow intuitive interaction with the robot. Users wear light-weight mixed reality goggles that integrate dis- plays and a camera, allowing the user to view and interact with the physical environment as well as with the virtual Bubblegrams interface linked to the robot's body. In order to eciently link Bubble- grams in real-time to the physical robot we imple- mented a vision algorithm based on Harr-like fea- tures which is the main topic of this paper. This paper briefly details the design of the Bubblegrams interface, the hardware and software we use for the current prototype, and the full details of the vision algorithm.