Remotion: A Motion-Based Capture and Replay Platform of Mobile Device Interaction for Remote Usability Testing

Remotion is an end-to-end system for capturing and replaying rich mobile device interactions, comprising both on-screen video and physical device motions. The blueprints and software provided here allow an interface to be instrumented with Remotion's capture and visualization system. Remotion is able to mimic mobile device motion through a software 3D graphical visualization and a robotic mount that replicates the movements of a mobile device from afar. Deployed together, experimenters can emulate the mobile device postures of a remote user as if they were in the room. This is important since many usability studies are carried remotely and the contribution and scale of those studies are irreplaceable. We compared how HCI experts ("analysts") observed remote users behavioral data across three replay platforms: a traditional live time-series of motion, Remotion's software visualization, and Remotion's hardware visualization. We found that Remotion can assist analysts to infer the user's attention, emotional state, habits, and active hand posture; Remotion also has a reduced effect on mental demand for analysts when analyzing the remote user's contextual information.