Adapting Evaluation To Study Behaviour In Context

We present a reflection on a series of studies of ubiquitous computing systems in which the process of evaluation evolved over time to account for the increasing difficulties inherent in assessing systems 'in the wild'. Ubiquitous systems are typically designed to be embedded in users' everyday lives, however, without knowing the ways in which people will appropriate the systems for use, it is often infeasible to identify a predetermined set of evaluation criteria that will capture the process of integration and appropriation. Based on our experiences, which became successively more distributed in time and space, we suggest that evaluation should become adaptive in order to more effectively study the emergent uses of ubiquitous computing systems over time.