Usability Evaluation of Handheld and Wearable AR devices: Exploring Collaboration and the Role of Physical Props

This work presents a study to evaluate the usability of handheld and wearable Augmented Reality (AR) devices for object manipulation in an individual and collaborative AR environment. We propose a scenario that incorporates physical props to foster an immersive AR experience, with tasks involving moving a virtual light source, changing the color of a virtual object anchored to the prop, and placing the object on the light. A controlled evaluation was performed that includes both quantitative and qualitative analyses. Two test sessions were conducted - an individual and a collaborative session. In the individual session, the participants completed the experiment using a smartphone and Magic Leap One AR headset, with tasks carried out via a touch screen on the smartphone and controller / hand gestures on the AR headset. Twelve participants were paired up to perform the same tasks in a shared AR space for the collaborative sessions. We performed statistical analysis on the collected data to provide insights into statistically significant results. The results demonstrated that though AR headsets are better suited for interactions that require spatial movements in the three-dimensional space, the difference in time taken to execute these interactions compared to a smartphone was not statistically significant. On the other hand, executing gestures on a smartphone was significantly faster compared to hand-based gestures using the AR headset.