An Enhanced Cardiac Auscultation Approach for the detection of abnormal Heartbeat and murmur based on Multimodal Deep Learning

Abstract Cardiovascular diseases are a leading cause of global mortality, and timely detection is essential to prevent their detrimental outcomes. Traditional diagnostic methods, such as segmentation, feature extraction, and subsequent heartbeat analysis, present inherent challenges. So far, the published deep learning methods have been designed for a single category, either finding abnormal signals or detecting murmur. Furthermore, most published deep learning methods used heartbeats recorded from different auscultations as similar, such that they are from the same origin. However, the flexibility in designing deep learning models made it possible to analyze heartbeats from various auscultation locations as distinct signals and categorize them in multiple labels. Therefore, this research proposed a novel multi-input-multi-output hybrid deep learning model for heartbeat analysis. This innovative approach integrated a Bidirectional Long Short Term Memory (BiLSTM) along with Dense layer as inputs and added UNet followed by Residual blocks as a backbone to find abnormal and murmured heartbeats simultaneously through multi-output architecture. The first four inputs used BiLSTM layers to extract temporal features independently from the four auscultation locations. The fifth input analyzed the patient's gender through the Dense layer and supported the model by focusing on gender while analyzing heartbeats. The UNet architecture transformed the temporal features into the required or more suitable format, and the Residual blocks managed the model's depth. Finally, all five networks combined and simultaneously classified the patient's heartbeats into two binary categories, providing information that either the signal was normal or the heartbeat had murmured. The Circor DigiScope dataset had missing features and, therefore, was unsuitable for experimentation. Thus, the dataset was modified by excluding samples lacking audio from essential auscultation sites. The proposed model, trained using the K-Fold cross-validation strategy, provided 95% accuracy and avoided overfitting on a small dataset. Therefore, the MIMO strategy provided an in-depth interpretation of cardiac signals by analyzing each signal separately through multiple perspectives. Consequently, it achieved better results and set the stage for future deep-learning algorithms in this domain. Furthermore, using less preprocessing and relying on the flexibility of deep learning algorithms underscored the model's potential to enhance diagnostic accuracy in real-world clinical scenarios.