Relevance-Based Feature Masking: Improving Neural Network Based Whale Classification Through Explainable Artificial Intelligence

Underwater sounds provide essential information for marine researchers to study sea mammals. During long-term studies large amounts of sound signals are being recorded using hydrophones. To facilitate the time consuming process of manually evaluating the recorded data, computational systems are often employed. Recent approaches utilize Convolutional Neural Networks (CNNs) to analyze spectrograms extracted from the audio signal. In this paper we explore the potential of relevance analysis to enhance the performance of existing CNN approaches. For this purpose, we present a fusion system that utilizes intermediate outputs of three state of the art CNNs, which are fine tuned to recognize whale sounds in spectrograms. Hereby we use Explainable Artificial Intelligence (XAI) to asses the relevance of each feature within the obtained representations. Based on those relevance values, we create novel masking algorithms to extract significant subsets of respective representations. These subsets are used to train an ensemble of classification systems that are serving as input for the final fusion step. We observe that a classification system can benefit from the inclusion of Relevance-based Feature Masking in terms of improved performance and reduced input dimensionality. The presented work is part of the INTERSPEECH 2019 Computational Paralinguistics Challenge.