FuSeR: Fusion of wearables data for StrEss Recognition using explainable artificial intelligence models

Wearables such as smartwatches have become increasingly popular and have many applications such as health monitoring and chronic disease management. As wearables are becoming more widely available and affordable, so personal data that these devices provide also increases. Additionally, many biological signal sensors are integrated into wearables and can be used for collecting personalised physiological data. Following data collection, classification techniques can aid in analysing the data insights. In this study, the effectiveness of a stress recognition approach using wearables’ physiological data fusion was evaluated. Specifically, four data level fusion methods (Fusion 2, 3, 4 and 5), consisting of two, three, four and five physiological modalities have been used. Different combinations from electrocardiogram (ECG), electrodermal activity (EDA), electromyography (EMG), accelerometer (Acc.), temperature (Temp.) and respiration (Resp.) signals have been considered for each fusion method to recognize stress. Extra gradient boost (XGBoost), light gradient boost (LGBoost) and CatBoost machine learning methods along with explainable artificial method (XAI) are implemented to classify stress. WESAD dataset is used for training and testing the models. Performance metrics for Fusion 1 (two physiological modalities), 2 (three physiological modalities), 3 (four physiological modalities) and 4 (five physiological modalities) are evaluated. Overall, the accuracy of the stress recognition model increases with increase in modalities. Fusion 4 consisting of five modalities provided the best performance with an accuracy from 95% to 99%. XAI explains more influence of EDA modality among other modalities on the model accuracy. The proposed stress recognition model with fusion of wearables data using XAI efficiently detects stress and contributes to mental health monitoring.