Machine-based emotion-assessment in waiting rooms – a feasibility and acceptance study

Background: Due to an aging society and changing health behaviors, emergency room crowding has become a major problem in western health care systems. Empowering patients and health care workers to assess necessary and relevant information is critical to streamline clinical workflows. Health kiosks, designed for services like self-check-in or (ideally contactless) health self-assessment may be instrumental in solving this issue. Based on the collected data, automated workflows such as flagging critical patients, inducing specific diagnostics or early symptomatic treatment could be implemented. Objective: Using an AI-supported software, which visually analyzes and categorizes facial expressions, the emotional status of hemato-oncologic patients in a German oncology outpatient clinic was examined. Additionally a survey was conducted, evaluating the acceptance of such a self-assessment solution. Results: 98% of the participants were not stressed by the real-time emotion analysis. However, the current set of registered emotion categories was found to be only partially sufficient to adequately describe the emotional status of the patients. More importantly, 88% of the participants found such a system to be meaningful. Also, 84% of the participants agreed that such a self-analysis could be of potential assistance. No relevant generation- or gender-specific differences could be observed. Discussion: Automated analysis of patients’ emotional status can be a first step toward a more comprehensive assessment of the respective health status. Patients, in particular the elderly, approve to the vision and development of such a system. Next steps are a further improvement of the AI-based emotion recognition software with respect to more emotional states as well as the definition, inclusion and ideally contactless acquisition of physical biomarkers (as e.g. heart rate or respiratory rate) determining physical and mental well-being.