Phase Aware Speech Enhancement Based on Depthwise Deep Convolutional Network

Deep neural network-based Speech Enhancement (SE) techniques aim to improve the clarity and quality of speech signals where neural models are trained to recover clean speech from noisy mixtures. Most of these techniques focus on estimating the magnitude of the spectrogram while reusing the phase component, which limits performance, especially under severe noise distortion. Additionally, high computational demands remain a major challenge for state-of-the-art deep learning models. This paper presents a framework called Depthwise Deep Convolutional Network (DDCN) method for single-channel speech enhancement, leveraging the time-frequency characteristics of speech signals through simultaneous phase and magnitude processing with reduced computations. By incorporating depthwise convolutions in both encoder and decoder architecture the model achieves enhanced computational efficiency. The DDCN SE system adopts parallel processing of the denoised real and imaginary spectrograms which correspond to the magnitude and phase information resulting in superior enhancement performance. The model is trained using a Signal-to-Distortion Ratio (SDR) loss function to minimize distortion and refine phase and magnitude estimation. This ensures that enhanced real and imaginary spectrograms closely match clean speech, improving spectral mapping, and refining the model’s spectral mapping capability. Experimental results demonstrate that the proposed DDCN model, with only 1.28 M parameters, achieves 45.42 FLOPS and RTF of 0.073 outperforms recent deep learning benchmarks, achieving superior speech quality and intelligibility. Specifically, when tested on the WSJ + DNS dataset, the DDCN model outperforms all baselines, achieving a notable SDR improvement of 2.14 dB over the best baseline, CTSNet, across various noise types. Additionally, DDCN surpasses advanced speech enhancement models like ZipEnhancer(S) and MP-SENetUp, achieving WB-PESQ of 3.74 and STOI of 97