From in the wild to in vivo: Video Analysis of Mobile Device Use

The explosion of mobile applications and services presents challenges for evaluation and user study. One successful approach has been to deploy instrumented applications, logging their use over long periods of time. We present an expansion of this by remotely recording video and audio of use, while also capturing device and app context. In vivo combines five data collection techniques -- screen recording, ambient audio recording, wearable cameras, data logging and distributed remote uploads. This data provides a range of insights and we discuss examples from previous work which reveal interaction design issues where interface confusions or task mismatches occur. We see how apps are integrated into ongoing activity and environment (such as how maps are used in situ), and how recorded conversations around and about apps may be used for evaluation purposes. We conclude by arguing that this combinative method helps us to move from considering app use in isolation, to studying app use in interaction.