Amphion

Amphion is a free, open-source toolkit for generating and evaluating speech, singing, and audio. It suits researchers and engineers who want to build or study audio systems in a Linux or self-hosted environment.

Its breadth and MIT license make it a flexible research foundation; text-to-music remains in development, and setup calls for technical expertise and NVIDIA-compatible infrastructure.

Overview

Amphion brings generation models, evaluation metrics, visualizations, and dataset preprocessing together for reproducible research. Supported tasks include text-to-speech, voice and accent conversion, singing voice synthesis, and text-to-audio. That range is valuable for teams exploring several audio-generation problems in one toolkit, but it is not a ready-made hosted voice service.

Installation is documented through Conda with Python 3.9.15 or Docker. The Docker path requires NVIDIA drivers, NVIDIA Container Toolkit, and CUDA, and the dataset must be mounted into the container. These demands are manageable for equipped research teams, but less suitable for people seeking a quick, hardware-agnostic workflow.

Key features

Speech and voice models

For text-to-speech, Amphion includes FastSpeech2, VITS, VALL-E, NaturalSpeech2, MaskGCT, and Vevo-TTS. Voice conversion models include Vevo, FACodec, and Noro; accent conversion is also supported. The variety gives developers multiple architectures to work with rather than locking them into one approach, though it also places Amphion firmly in the toolkit category.

Controllable singing and text-to-audio

Vevo2 supports controllable speech and singing generation, including voice conversion, singing voice conversion, singing voice editing, singing style conversion, and melody control. The controls make it relevant to work spanning speech and singing, not only standard voice conversion. Text-to-audio uses a latent diffusion model and implements the text-to-audio component of Amphion's NeurIPS 2023 paper.

Text-to-music is still in development. Readers whose main requirement is generating music directly from prompts should choose a tool with that task supported rather than plan around Amphion.

Evaluation and datasets

Amphion supplies objective metrics for F0 and energy modeling, intelligibility, spectrogram distortion, and speaker similarity. Its interactive visualizations cover classic model processing, including SingVisio for the diffusion model used in singing voice conversion. Together, these tools support examining and comparing model behavior, not just generating outputs.

Preprocessing is unified across multiple open-source audio datasets, with support for the Emilia dataset and Emilia-Pipe pipeline for in-the-wild speech. Content-based features can draw on pretrained WeNet, Whisper, and ContentVec models.

Pricing

Amphion costs nothing. The MIT-licensed toolkit is free for research and commercial use, so teams can explore or use it commercially without a paid tier or per-seat plan. Its cost advantage does not remove the practical price of providing compatible hardware and managing setup.

Platforms

Amphion is available for Linux and self-hosted use. Audio uploads support WAV, MP3, and FLAC, and exports are in WAV. The project documents Conda and Docker setup; Docker requires NVIDIA drivers, NVIDIA Container Toolkit, CUDA, and a mounted dataset. That makes it a fit for Linux-based development environments, not a turnkey cross-platform desktop or web app.

Who it's for

Amphion is aimed at researchers and engineers developing audio, music, and speech-generation systems, including junior practitioners entering the field. It is especially useful for teams that need model implementations, evaluation, visualizations, and dataset preparation in one open-source project. People who want a simple voice-conversion app, or need text-to-music today, should look elsewhere.

Pros and cons

• Pros: Broad support for speech, voice conversion, singing, and text-to-audio gives research teams one toolkit for several generation tasks.
• Pros: Evaluation metrics, visualizations, and dataset preprocessing complement generation models for reproducible research workflows.
• Pros: MIT licensing and free research and commercial use make it accessible for commercial as well as academic projects.
• Cons: Text-to-music is in development, so it does not meet that need as a supported capability.
• Cons: The documented Docker path requires NVIDIA software and dataset mounting, creating a substantial setup hurdle for casual users.
• Cons: Linux and self-hosted availability favors technical teams over users seeking a hosted, cross-platform voice app.

Alternatives

LALAL.AI is worth considering for a freemium option spanning web, desktop, mobile, and Linux; its free Starter plan limits uploads to 200 MB per file and the Relaxed Queue to 10 minutes, with previews but no full downloads.

Resemble AI is a broader platform alternative across web, desktop, mobile, API, and self-hosted environments. Its Flex plan starts free with one team seat and pay-as-you-go credits that never expire.

Altered Studio may suit users seeking a desktop or web voice tool: its free plan includes three minutes per month of Voice Morphing, 10,000 AI Tokens, and local voice cloning, with attribution and noncommercial restrictions.

Applio is another free, MIT-licensed open-source option for voice conversion, with Linux, macOS, Windows, web, and self-hosted availability.

IK Multimedia ReSing fits users who prefer a macOS or Windows application; its free plan includes two voices, two instruments, and one RVC import, but no model generation.

Audimee is a web-based alternative with a free plan that allows 15 minutes of conversions and 11 royalty-free voices, but no custom voice model slots and MP3 downloads only.

AudioPod AI is an API, iOS, and web option whose free Basic plan includes 1,000 credits per month, approximately three minutes of text-to-speech and one minute of music generation.

Kits AI is an alternative for users seeking a web, Windows, or API workflow; its free plan includes 15 conversion minutes, voice designer/blender, and generative vocals, but no voice slots or download minutes.

The AI Voice Conversion Software list and Text-to-Speech Software list can help compare other tools in those categories.

Verdict

Choose Amphion if you are a researcher or engineer who wants a free, MIT-licensed foundation for studying or developing multiple speech, singing, and audio-generation tasks, with evaluation and dataset tools alongside models. Its strongest reason to choose it is that research breadth; its clearest reason to look elsewhere is the technical setup, compounded by text-to-music not yet being supported.