Heuristic evaluation and user tests of wearable mobile health monitoring applications - what results do different methods yield?

Wearable smart bands connected to mobile apps in smartphones provide essential resources for health and activity monitoring. Evaluating this kind of application has many challenges, considering the mobile and wearable nature. This paper presents a comparative study employing heuristic evaluations and user tests of three health monitoring apps used with their wearable smart bands, to analyse the types of problems identified by each method. The study examined the Mi Band 2 with the MiFit app, the MTR-01 with the DayDay Band app, and Goral Y5 with the Lefun app. The apps and wearable smart bands were evaluated by three inspectors and employing user tests in a laboratory with a panel of 11 users simulating a short walk. After mapping similar problems encountered, the study found 18 problems by both methods, 27 problems encountered only by users, and 105 problems encountered only by the heuristic evaluations. The results in this study are relevant to inform designers about the types of problems in wearable health apps and devices using different methods, and research gaps to improve usability methods and resources to strengthen evaluations in this domain.