StreamVC: Real-Time Low-Latency Voice Conversion
CoRR(2024)
摘要
We present StreamVC, a streaming voice conversion solution that preserves the
content and prosody of any source speech while matching the voice timbre from
any target speech. Unlike previous approaches, StreamVC produces the resulting
waveform at low latency from the input signal even on a mobile platform, making
it applicable to real-time communication scenarios like calls and video
conferencing, and addressing use cases such as voice anonymization in these
scenarios. Our design leverages the architecture and training strategy of the
SoundStream neural audio codec for lightweight high-quality speech synthesis.
We demonstrate the feasibility of learning soft speech units causally, as well
as the effectiveness of supplying whitened fundamental frequency information to
improve pitch stability without leaking the source timbre information.
更多查看译文
关键词
Voice conversion,On-device neural audio processing,Real-time voice changer
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要