Robust Video Matting

Robust Video Matting is a free project for isolating people from video, aimed chiefly at developers and technically confident creators building matting into a workflow. Its live processing and broad model exports make it a flexible starting point; it is less suited to users who want a polished, turnkey editor or expect fast file conversion.

Overview

The project uses a recurrent neural network with temporal memory to process video frames, and supports background replacement at resolutions up to 4K. Its focus is human matting, not general video editing, so its strongest fit is a workflow that needs people separated from their backgrounds rather than an all-purpose clip editor.

The source code and pretrained models are released under GPL-3.0. The project was developed at ByteDance Inc. Teams planning to incorporate or distribute the code should consider the license as part of that decision.

Key features

• Matting without extra inputs: The project says it can process video in real time without additional inputs. That can simplify live or continuous footage workflows where supplying guidance for individual frames would be cumbersome.
• Model variants: MobileNetv3 is recommended for most use cases. ResNet50 is larger and offers small performance improvements, so it is a narrower choice for users who value those gains enough to accept a larger model.
• Multiple model formats: Models are provided for PyTorch, TorchHub, TorchScript, ONNX, TensorFlow, TensorFlow.js, and CoreML. This range gives developers options for different runtimes, though framework support does not remove the need to check each runtime's constraints.
• Browser and Colab demos: The webcam demo runs the model live in a browser and can visualize recurrent states. A Colab demo lets users try their own videos with a free GPU, making evaluation possible without first arranging local GPU hardware.
• Video conversion: The conversion API accepts a video file or image sequence and can produce a composited video or PNG sequence, with optional alpha and foreground outputs. The project cautions that conversion runs much slower than the reported tensor throughput because it lacks hardware video encoding and decoding and parallel tensor transfers.
• Performance figures: The README reports 172 FPS at HD and 154 FPS at 4K on an RTX 3090 with FP16, batch size 1, and frame chunk 1. It also reports 104 FPS at HD and 76 FPS at 4K on an Nvidia GTX 1080 Ti. These are model throughput figures, not a promise that the file-conversion script will process footage at the same rate.

Inference, training, and evaluation documentation provides guidance for using and developing the model. ONNX models use opset 12 and are tested with CPU and CUDA backends.

Pricing

Robust Video Matting — free includes pretrained models and source code under GPL-3.0. There are no paid tiers or quotas in this plan. Its cost advantage suits developers and creators who can work with the project directly; the trade-off is that this is a model and conversion workflow, not a consumer editing subscription.

Platforms

The project is associated with Android, API, iOS, Linux, self-hosted, and web use. CoreML models require iOS 13 or later and do not support dynamic resolution. Android support is linked through a third-party NCNN C++ project, so users should distinguish that route from the project's provided model exports.

Who it's for

Choose Robust Video Matting if you need human video matting in a custom workflow, want to select among model formats, or need to evaluate recurrent states in a live browser demo. It is also a sensible option for users who can test their footage in Colab before integrating the model.

Look elsewhere if your priority is a ready-made editing interface or predictable high-speed batch conversion: the project emphasizes model capabilities, and its conversion script is explicitly slower than raw tensor throughput. Mobile developers should also account for the iOS resolution constraint and the third-party nature of the Android route.

Pros and cons

• Pro: Free source code and pretrained models make experimentation and integration possible without a subscription.
• Pro: Multiple framework formats and browser and Colab demos support evaluation across different development workflows.
• Pro: The project reports real-time throughput, including 4K figures, and says no additional inputs are needed.
• Con: File conversion is expected to be much slower than tensor throughput because video encoding, decoding, and parallel transfers are not hardware-accelerated in the script.
• Con: CoreML does not support dynamic resolution, while the Android option relies on a third-party project.

Alternatives

For a broader set of background-removal choices, see Video Background Removal Software or AI Video Background Removers.

• Cutout.Pro is a freemium option with Android, API, iOS, Linux, macOS, web, and Windows platforms; consider it if you want to compare a subscription-based alternative.
• BackgroundRemover is a free, open-source project that can be installed locally with pip or from source; choose it when local installation and a free workflow are priorities.
• Bria Video Background Removal offers a free tier with 100 generations, cloud-only deployment, a 10-requests-per-minute limit, queue-based latency, and Discord community support; it may fit users who prefer a hosted service with a defined free allowance.
• Pixelcut offers limited free background removal and free exports without a watermark; consider it when those editing and export terms are a better fit.
• VCam has a free Starter plan with a watermark and up to 500MB storage; it may suit users comparing a workspace-based option.
• EZ-CorridorKey is a free desktop app for Linux, macOS, and Windows; consider it if you want a desktop alternative.
• Knockout API provides a free recurring allowance of 30 images per month; it may be worth comparing for image-focused API use.
• Transparent Background is an open-source Python package with GUI and webcam support; consider it as another free desktop-oriented option.

Verdict

Robust Video Matting is a strong choice for developers and technically confident creators who need free, adaptable human matting and can work with model exports or conversion tools. Its main draw is the combination of real-time processing claims, multiple frameworks, and usable demos; look elsewhere if you need a consumer-ready editor or fast file conversion without additional workflow work.