Improving GAN-based Vocoder for Fast and High-Quality Speech Synthesis

Following tremendous success in the Generative Adversarial Network(GAN), the GAN-based vocoders have recently shown much faster speed in waveform generation. However, the quality of generated speech is slightly inferior, and the real-time factor (RTF) still can't be satisfied in many devices with limited resources. To address the issues, we propose a new GAN-based vocoder model. Firstly, we introduce the Shuffle-Residual Block into the generator to get a lower RTF. Secondly, we propose a Frequency Transformation Block in the discriminator to capture the correlation between different frequency bins in every frame. To the best of our knowledge, our model achieves the lowest RTF of the GAN-based vocoders under the premise of ensuring the speech quality. In our experiments, our model shows a lower RTF with more than 40% improvement and higher speech quality than MB-MelGAN and HiFi-GAN V2.