UBeBot: voice-driven, personalized, avatar-based communicative video content in A/R

UBeBot allows a mobile user to create a 3D avatar of themselves using a photo, as well as dress and style the avatar. Users then record their voice, allowing the avatar to act our the content of the utterance, including lip sync, facial expressions, gestures and other body language. This animated performance is generated automatically by analyzing the recorded voice signal, and does not require any camera tracking. The 3D avatar can then be placed in augmented reality (A/R) and saved to a video for sharing on social media. A mobile user is thus able to create their own personalized, animated and voiced 3D A/R content. Performances can be saved and triggered to create the appearance of interactive conversations.