DiffSinger

What DiffSinger is

DiffSinger is a self-hosted system for synthesizing singing voices with a shallow diffusion mechanism. Its maintained version adapts synthesized audio to a 44.1 kHz sampling rate, and the project includes improved acoustic models and accelerated diffusion sampling. Variance models and parameters can predict and control pitch, energy, breathiness, and other vocal aspects. The linked MakeDiffSinger project provides dataset-building pipelines and tools for slicing and labeling recordings. The getting-started guide covers dataset preprocessing, model training, and inference on DS files using variance or acoustic models. DiffSinger uses ONNX for deployment and links to OpenUtau and DiffScope for production or deployment; DiffScope is described as an editor under development. Users prepare their own data and assets. Installation requires Python 3.10 or later, with PyTorch 2.4.0 or later recommended. The repository prohibits generating a person's voice without their consent; its guide also includes a tool to remove speaker embeddings from checkpoints before model distribution. The project points users to tutorials, GitHub issues and discussions, and QQ and Discord groups for support.

Who it suits

It suits singing-voice synthesis practitioners and production teams preparing their own data and assets. The project describes its functionality as designed for production deployment and the singing voice synthesis community.

Limits

The free plan requires users to prepare their own data and assets. Installation requires Python 3.10 or later, and PyTorch 2.4.0 or later is recommended.

Getting started

1. Open the DiffSinger repository and follow its getting-started guide.
2. Install Python 3.10 or later; PyTorch 2.4.0 or later is recommended.
3. Prepare a dataset, using MakeDiffSinger tools for recording slicing and labeling if useful.
4. Preprocess the dataset and train a model.
5. Run inference on DS files with a variance or acoustic model.
6. Deploy using ONNX or explore the linked production projects.

Verdict

DiffSinger offers a configurable synthesis workflow, dataset tools, and ONNX deployment for users prepared to supply their own assets. Check the Python requirement and consent restriction before starting.