EzAudio

Paid Self-hostedWeb

What EzAudio is

EzAudio is an open-source model that generates audio from text prompts, with additional support for editing and inpainting. Its ControlNet demo uses a text prompt and an audio reference to generate audio. The model operates in the latent space of a one-dimensional waveform variational autoencoder. The paper describes an optimized diffusion transformer intended to improve convergence speed, training stability, and memory use, along with classifier-free guidance rescaling for prompt alignment at higher guidance scores. Its training approach has three stages: unlabeled audio, machine-generated captions, and human-labeled fine-tuning data. The project links browser demos for EzAudio and EzAudio-ControlNet, and publishes code and model checkpoints for local use. Example code uses PyTorch and SoundFile, with checkpoints hosted on Hugging Face. Local installation requires cloning the repository and installing Python dependencies; the usage example selects CUDA when available and otherwise uses CPU. EzAudio is not deployed by an inference provider. The GitHub repository identifies an MIT license, while the project website lists a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license. The training pipeline and dataset are listed as todo items.

Who it suits

EzAudio suits people exploring text-to-audio generation, audio editing, or inpainting through its browser demos or local code. Local use involves installing Python dependencies and choosing an available compute option in the example.

What it does well

Supports text-to-audio generation, editing, and inpainting.
ControlNet demo accepts an audio reference.
Browser demos are linked for both versions.
Code and model checkpoints are published.
Usage example falls back to CPU when CUDA is unavailable.

What to know first

Not deployed by an inference provider.
Local use requires cloning the repository and installing dependencies.
Training pipeline and dataset remain todo items.
Commercial use is unclear.

Limits

The repository's automatic checkpoint downloading and release of the training pipeline and dataset are still todo items.

Getting started

Open the EzAudio project website.
Choose a linked EzAudio or EzAudio-ControlNet browser demo, or use the published local code and checkpoints.
For local use, clone the repository and install its Python dependencies.
Follow the usage example, which selects CUDA when available and otherwise uses CPU.

Verdict

EzAudio offers several audio generation and editing workflows through demos and local code. It is not hosted by an inference provider, and the project lists the training pipeline and dataset as unfinished todo items.

Questions

Is EzAudio free?

The pricing model is free, and the project is described as open source.

What can EzAudio do?

It generates audio from text prompts and also describes audio editing and inpainting. Its ControlNet example uses both a text prompt and an audio reference.

Can I try it in a browser?

The repository links to Hugging Face Spaces demos for EzAudio and EzAudio-ControlNet.

Can I run it locally?

The project publishes code and model checkpoints for local use. Setup requires cloning the repository and installing Python dependencies.

What license applies?

The GitHub repository identifies an MIT license, while the project website identifies a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license.

Is commercial use allowed?

Commercial use is unclear.

Facts from EzAudio's own pages

Access: The maker links to browser demos for EzAudio and EzAudio-ControlNet, and publishes code and model checkpoints for local use. · github.com, 9 Oct 2026
Audio editing: The project describes text to audio generation, audio editing, and inpainting. · github.com, 9 Oct 2026
Authors and affiliations: The project page lists Jiarui Hai and coauthors, affiliated with Johns Hopkins University and Tencent AI Lab. · haidog-yaqub.github.io, 9 Oct 2026
Availability limitation: The model page says EzAudio is not deployed by an inference provider. · huggingface.co, 9 Oct 2026
ControlNet: The repository includes a ControlNet demo and example code that generates audio using a text prompt and an audio reference. · github.com, 9 Oct 2026
Editing and inpainting: The project describes support for audio editing and inpainting in addition to text-to-audio generation. · github.com, 9 Oct 2026
Efficiency: The paper describes an optimized diffusion transformer designed to improve convergence speed, training stability, and memory usage. · haidog-yaqub.github.io, 9 Oct 2026
Founded: 2024 · haidog-yaqub.github.io, 28 Sept 2026
Integrations: The repository's example code uses PyTorch and SoundFile, and links to model checkpoints hosted on Hugging Face. · github.com, 9 Oct 2026
License: The GitHub repository identifies its license as MIT. · github.com, 9 Oct 2026
Local requirements: The usage example selects CUDA when available and otherwise uses CPU; installation requires cloning the repository and installing its Python dependencies. · github.com, 9 Oct 2026
Model design: EzAudio uses an optimized diffusion transformer for audio latent representations and a one-dimensional waveform variational autoencoder. · arxiv.org, 9 Oct 2026
Online demos: The repository links to Hugging Face Spaces for EzAudio and EzAudio-ControlNet demos. · github.com, 9 Oct 2026
Project page license: The project website identifies its license as Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International. · haidog-yaqub.github.io, 9 Oct 2026
Prompt alignment: The model uses classifier free guidance rescaling to support stronger prompt alignment while preserving audio quality at higher guidance scores. · haidog-yaqub.github.io, 9 Oct 2026
Prompt guidance: The paper describes classifier-free guidance rescaling to improve prompt alignment while preserving audio quality at higher guidance scores. · arxiv.org, 9 Oct 2026
Purpose: EzAudio is a diffusion based model that generates audio from text prompts. · haidog-yaqub.github.io, 9 Oct 2026
Release status: The README lists automatic checkpoint downloading and release of the training pipeline and dataset as todo items. · github.com, 9 Oct 2026
Training: The paper describes a three stage training approach using unlabeled audio, machine generated captions, and human labeled data for fine tuning. · haidog-yaqub.github.io, 9 Oct 2026
Training resources: The repository's todo list includes releasing the training pipeline and dataset. · github.com, 9 Oct 2026

Where EzAudio stands on Freedom251

AI Sound Effect Generators#28 of 29