- Research affiliation
- Xiaomi Corp., as listed in the paper
- Language coverage
- More than 600 languages
- Output sample rate
- 24 kHz
- Interfaces
- Python, CLI, Gradio; self-hosted
Strengths worth exploring
Built for broad multilingual coverage
The paper describes a non-autoregressive diffusion-language-model approach that predicts multi-codebook acoustic tokens directly, simplifying the usual two-stage generation path.
Official source [2]Save and reuse a voice prompt
Create a cloning prompt once, save it, and load it for later scripts. This avoids repeating reference loading and automatic transcription for a recurring narrator or character.
Official source [1]Control pronunciation and vocal cues
Chinese pinyin, English phoneme annotations, and non-verbal symbols provide targeted controls. Speed and a requested output duration give further ways to shape pacing.
Official source [1]Cloning, design, and automatic voices
Cloning accepts reference audio and its transcript; Whisper can transcribe when text is omitted. Voice design instead combines attributes such as gender, age, pitch, and accent.
The project identifies cloning as its main training task. Voice design is trained on Chinese and English data, so its reliability should be assessed separately from multilingual cloning.
Official source [1]Cross-language cloning can carry an accent
The project recommends a clear 3–10 second reference. When the reference and target languages differ, the output can carry the source-language accent. Use a same-language reference when standard pronunciation is the priority.
Official source [1]Usage and deployment
- Prepare Git, uv, and an appropriate PyTorch runtime. Official instructions cover NVIDIA, Apple Silicon, and Intel XPU environments.
- Clone the repository, sync dependencies, and launch Gradio with the commands below. The first run downloads the k2-fsa/OmniVoice checkpoint.
- Test a short cloned passage before moving to batch CLI or Python workflows. An accurate reference transcript makes pronunciation issues easier to diagnose.
git clone https://github.com/k2-fsa/OmniVoice.git
cd OmniVoice
uv sync
uv run omnivoice-demo --ip 127.0.0.1 --port 8001Before you get started
- Coverage of 600+ languages does not mean identical pronunciation quality, fidelity, or accent handling in every language.
- Published speed measurements have specific hardware and inference settings; they are not fixed completion-time promises for other machines.
Where it fits
A strong candidate for multilingual localization, low-resource language exploration, reusable voices, and teams operating their own inference environment.
Official resources and sources
DS Speech currently provides Qwen-Audio-3.0-TTS online. Other models in this library are documented for learning and self-deployment.