Experiencing Real-time 3D Interaction with Depth Maps for Mobile Augmented Reality in DepthLab

ABSTRACTWe demonstrate DepthLab, a playground for interactive augmented reality experiences leveraging the shape and depth of the physical environment on a mobile phone. Based on the ARCore Depth API, DepthLab encapsulates a variety of depth-based UI/UX paradigms, including geometry-aware rendering (occlusion, shadows, texture decals), surface interaction behaviors (physics, collision detection, avatar path planning), and visual effects (relighting, 3D-anchored focus and aperture effects, 3D photos). We have open-sourced our software at https://github.com/googlesamples/arcore-depth-lab to facilitate future research and development in depth-aware mobile AR experiences. With DepthLab, we aim to help mobile developers to effortlessly integrate depth into their AR experiences and amplify the expression of their creative vision.