SELM: Speech Enhancement Using Discrete Tokens and Language Models
ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)(2023)
摘要
Language models (LMs) have shown superior performances in various speech
generation tasks recently, demonstrating their powerful ability for semantic
context modeling. Given the intrinsic similarity between speech generation and
speech enhancement, harnessing semantic information holds potential advantages
for speech enhancement tasks. In light of this, we propose SELM, a novel
paradigm for speech enhancement, which integrates discrete tokens and leverages
language models. SELM comprises three stages: encoding, modeling, and decoding.
We transform continuous waveform signals into discrete tokens using pre-trained
self-supervised learning (SSL) models and a k-means tokenizer. Language models
then capture comprehensive contextual information within these tokens. Finally,
a detokenizer and HiFi-GAN restore them into enhanced speech. Experimental
results demonstrate that SELM achieves comparable performance in objective
metrics alongside superior results in subjective perception. Our demos are
available https://honee-w.github.io/SELM/.
更多查看译文
关键词
speech enhancement,language models,generative model,staged approach
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要