Analysis of user interactions with a digital health wallet for enabling care continuity in the context of an ongoing pandemic

Background The onset of COVID-19 and related policy responses made it difficult to study interactive health informatics solutions in clinical study settings. Instrumented log and event data from interactive systems capture temporal details that can be used to generate insights about care continuity during ongoing pandemics. Objective To investigate user interactions with a digital health wallet (DHW) system for addressing care continuity challenges in chronic disease management in the context of an ongoing pandemic. Materials and methods We analyzed user interaction log data generated by clinicians, nurses, and patients from the deployment of a DHW in a feasibility study conducted during the COVID-19 pandemic in Kenya. We used the Hamming distance from Information Theory to quantify deviations of usage patterns extracted from the events data from predetermined workflow sequences supported by the platform. Results Nurses interacted with all the user interface elements relevant to triage. Clinicians interacted with only 43% of elements relevant to consultation, while patients interacted with 67% of the relevant user interface elements. Nurses and clinicians deviated from the predetermined workflow sequences by 42% and 36%, respectively. Most deviations pertained to users going back to previous steps in their usage workflow. Conclusions User interaction log analysis is a valuable alternative method for generating and quantifying user experiences in the context of ongoing pandemics. However, researchers should mitigate the potential disruptions of the actual use of the studied technologies as well as use multiple approaches to investigate user experiences of health technology during pandemics.