Running yt-dlp and FFmpeg well from a terminal is a different job from running them inside an asynchronous worker. Once downloads move behind a queue, the usual failure points are a worker that cannot find the FFmpeg binary, file paths that resolve differently than they do in a shell, post-processing errors hidden behind a job that looks successful, and queue retries or redelivery that run the same media work twice. Fixing these means making the boundaries between the downloader, the FFmpeg subprocess, the filesystem, and the queue explicit.
This guide maps those failure modes rather than reconstructing a single incident log. Each one is a mechanism described in the yt-dlp or Celery project documentation, paired with a check you can run to see whether it applies to your setup. Celery behavior is quoted from the Celery 5.6.3 documentation; confirm settings against your own broker and version before relying on them.
As an Amazon Associate I earn from qualifying purchases.
Where the boundaries sit
A single queued media job crosses five boundaries. Each can fail on its own, and each fails differently once the work leaves your shell.
| Boundary | What it owns | How it fails after the move to a worker |
|---|---|---|
| Worker runtime | Executable lookup, environment variables, working directory | The binary works in your shell but is absent from the worker image or the service user’s search path |
| yt-dlp | Extraction, download, format selection, post-processor dispatch | The download stage completes, then a post-processor fails |
| FFmpeg and ffprobe | Merging separate streams, conversion, metadata work | A non-zero exit that nothing records |
| Filesystem | Output directories, file names, partial files | Relative paths resolve against a different directory; half-written files look complete |
| Queue | Delivery, acknowledgement, time limits, concurrency | Duplicate execution, forced process kills, short jobs waiting behind long ones |
1. The worker cannot find FFmpeg
The yt-dlp README describes ffmpeg and ffprobe as strongly recommended dependencies, required for merging separate video and audio streams and for several post-processing operations. The dependency is the FFmpeg binary itself. A Python package with a similar name does not provide it, so a successful pip install tells you nothing about what the worker can execute. (yt-dlp project README)
#1 Best Overall
- 【1080P HD High Quality】Capture resolution up to 1080p for video source and it is ideal for all HDMI devices such as PS4, PS3, Xbox One, Xbox 360, Wii U, DVDs, DSLR, Camera, Security Camera and set top box. Note: Video input supports 4K30/60Hz and 1080p120/144Hz. Does not support 4K120Hz/144Hz. Output supports up to 2K30Hz.
- 【Plug and Play】No driver or external power supply required, true PnP. Once plugged in, the device is identified automatically as a webcam. Detect input and adjust output automatically. Won't occupy CPU, optional audio capture. No freeze with correct setting.
- 【Compatible with Multiple Systems】suitable for Windows and Mac OS. High speed USB 3.0 technology and superior low latency technology makes it easier for you to transmit live streaming to Twitch, Youtube, Facebook, Twitter, OBS, Potplayer and VLC.
- 【HDMI LOOP-OUT】Based on the high-speed USB 3.0 technology, it can capture one single channel HD HDMI video signal. There is no delay when you are playing game live.
- 【Support Mic-in for Commentary】Rybozen capture card has microphone input and you can use it to add external commentary when playing a game. Please note: it only accepts 3.5mm TRS standard microphone headset.
Check the runtime the worker actually uses, not your developer shell:
- Open a shell inside the worker environment. In a container, run
docker exec -it <worker-container> sh. For a systemd service, runsudo -u <service-user> -H sh. - Run
ffmpeg -versionandffprobe -version. Expected result: a version banner for each. A “not found” error means the binary is not on that runtime’s search path. - Run the same lookup through Python, because Python is the process that launches the subprocess:
python -c "import shutil; print(shutil.which("ffmpeg"), shutil.which("ffprobe"))". Expected result: two absolute paths. ANoneresult means the worker will not find the binary even if your shell does.
The fix is to install the binary into the worker image or onto the host, on a search path the worker process inherits. Restart the workers afterward, because a running process keeps the environment it was started with.
2. Paths that worked interactively resolve somewhere else
A relative output path depends on the worker’s working directory, which the process manager sets, not you. yt-dlp’s -P option specifies where each file type is saved, and -o sets the output template. The yt-dlp FAQ recommends these options for directing downloads to a folder. (yt-dlp README: output templates and options; yt-dlp FAQ)
Give each job its own absolute directory and let the template supply the extension:
Rank #2
- 【1080P 60FPS Video Capture Card】 This HDMI game capture card is based on USB3.0 high speed transmission port, input resolution up to 4K@30HZ, output resolution up to 2K@30Hz or 1920×1080@60Hz. Type c and USB interface can meet most of the devices in daily life. Easily meet the online capture, real-time recording, online meetings, live gaming and other functions, so you have a better visual enjoyment. Note: For capture use only; requires capture software to function and is not intended for direct screen casting to a monitor or TV
- 【Ultra Low Latency Screen Sharing】 HDMI capture card is made of good quality aluminum alloy with strong heat dissipation, allowing you to enjoy ultra low latency while live gaming or video recording or live streaming, avoiding blue screens and lag. This HDMI to USBC capture card supports easy recording of good quality audio or HD video and transferring it to your computer or streaming platform, allowing you to record 60 fps HD video directly on your hard drive and real-time preview
- 【Plug and Play, Easy to Carry】 This HDMI 1080P video capture card does not require any additional drivers or external power supply, just plug and play for fast capture. The capture card is small and lightweight, so you can put it in your bag for emergencies, making it very portable for outdoor live streaming. It's also a great way to share content in game recording, video conference, video recorder and online teaching
- 【Wide Compatibility USB Capture Card】 Easily streams to Facebook, Youtube or Twitch. With the connection, this HDMI to USB C/3.0 video capture devices can be working on several Operating Systems and various software: Windows 7/ 8/ 10, Mac OS or above, Linux, Android, Laptop, Xbox One, PS3/PS4/PS5, Camera, DVDs, Set Top Box, Webcame, DSLR, Switch/Switch 2, TV BOX, HDTV, Potplayer/VLC, ZOOM, OBS Studio etc.
- 【Package Content & Note】 1x HD Audio Capture Card , 1x USB 3.0 to USB C Adapter (A-side 3.0, B-side 2.0), 1x user manual. Please note that you need to restart the OBS Studio software after the audio setup is complete, otherwise it will result in no sound output. When using an adapter, if the device is recognized as USB 2.0, try using the other side with the USB-C port. Simply flip the capture card and reconnect it to be recognized as USB 3.0
yt-dlp -P "/srv/media/jobs/7f3a2c" -o "%(id)s.%(ext)s" "https://example.com/video/123"The README warns that hard-coding a file extension is not recommended because it can break post-processing, so %(ext)s is the safer choice.
- Confirm the worker user can write to the directory:
test -w /srv/media/jobs && echo writable. - Log the absolute path once the download finishes, and compare it with the path every later stage reads. A mismatch produces the familiar “output missing” error even though the file exists somewhere else.
3. The download looks finished, but post-processing failed
yt-dlp runs FFmpeg as a subprocess. Its FFmpeg postprocessor raises an error when the return code falls outside the set it expects. The exec postprocessor likewise turns non-zero return codes into post-processing errors. (yt-dlp FFmpeg postprocessor source; yt-dlp exec postprocessor source)
The failure pattern is a wrapper that asks only whether an output file appeared. An intermediate file, such as a separately downloaded video or audio stream, can exist even though the merge never completed, so a status check based on file presence can mark an incomplete job as successful. Wrap the call so the following are preserved:
Free tools Windows power users keep installed
One-click scans. No signup required.
- The return code and the tail of stderr, stored with the job record.
- The stage that failed (download, merge, or conversion), so the failure is attributable.
- A final check that the expected output exists and is non-empty before the job is marked complete. Running
ffprobeon it is a stronger check.
import subprocess
result = subprocess.run(
["yt-dlp", "-P", job_dir, "-o", "%(id)s.%(ext)s", url],
capture_output=True,
text=True,
timeout=1800,
)
if result.returncode != 0:
raise RuntimeError(f"yt-dlp exited {result.returncode}: {result.stderr[-2000:]}")The timeout argument raises subprocess.TimeoutExpired rather than a return code, so catch it separately and clean up the job directory before reporting the failure.
Rank #3
- AV TO USB Converter: Capture videos and audios from VHS, VCR, Hi8, DV tapes to a PC, with the help of our USB Video Converter. Save room while digitizing your favorite old memories
- Quality Capture Card: Our USB Video Capture Card converting anolog RCA composite input into HD 720P USB output and capturing audio without any sound card. Advanced signal processing technology provides you with great precision, colors, resolutions, and details.
- Plug and Play: Automatically install the driver once you hook up this RCA to USB Converter to a PC. No external power is needed. User-friendly and easy to operate
- Wide Compatibility: The Video Capture Card can work with video devices with RCA connector or S-Video connector, such as VHS, VCR, Hi8, camcorder, compatible with Windows and Mac OS. Support video formats like NTSC, PAL, and support brightness, contrast, hue, and saturation control
- Note: The Video Converter is used with acquisition software. We recommend OBS Studio or PotPlayer for Windows, and QuickTime Player for Mac. They can be downloaded for free online. Please operate according to the steps in User Manual or contact us if you have any questions
4. Retries and redelivery run the same media job twice
Three Celery behaviors determine what happens when a worker dies or a task fails. All three are described in the Celery 5.6.3 Tasks guide. (Celery Tasks guide)
Early acknowledgement is the default
Celery acknowledges tasks before execution by default, because it cannot know whether a task is idempotent. With late acknowledgement (task_acks_late), the task is acknowledged after it runs, so a worker crash in the middle can cause the task to execute again. The guide states the requirement this way: “Ideally task functions should be idempotent: meaning the function won’t cause unintended effects even if called multiple times with the same arguments.”
The reverse also matters. Under early acknowledgement, a job the worker received and then lost in a crash is not redelivered by the broker, so a missing job record is a symptom to check for, not a sign the job ran.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Scan for outdated or missing drivers - takes under a minute3Repair Windows errors before they cause bigger problemsExceptions are handled differently from redelivery
An exception raised inside a task is treated as a normal task outcome. For recoverable failures, such as a transient network error, the documented mechanism is retry(), so put the retry policy in the task. Reserve retries for transient failures. A missing binary or a corrupt source will fail identically on every attempt.
Rank #4
- High-Quality Video Capture, 4K HDMI Capture Card Ready: Capture smooth and vibrant video with this 4K HDMI capture card, engineered for gamers and content creators who demand crisp 1080P 60FPS video quality. Whether you're streaming to Twitch or recording gameplay for YouTube, your footage will look professional and detailed
- Plug-and-Play USB Capture Card, No Drivers Needed: Designed as a USB capture card for streaming, this device works instantly out of the box, just plug into your PC or laptop and start capturing. Fully compatible with popular software like OBS Studio, Streamlabs, and XSplit, making setup quick and stress-free for beginners and pros alike
- Universal Compatibility PS5, Xbox, Switch & More: Stream or record gameplay from virtually any HDMI-enabled device including Nintendo Switch, PS5, Xbox Series X, DSLR cameras, and PCs. The video capture card for gaming supports seamless passthrough so you can play without lag while your audience watches every frame in real time
- Low-Latency Performance for Smooth Streaming: This capture card for streaming minimizes delay between gameplay and broadcast, so you get reliable, low-latency capture that works well for competitive gaming, live broadcasts, and podcast sessions. Suitable for those building their channel with high-quality, engaging content
- Compact & Portable Design for Content Creators: Lightweight and portable, this USB 3.0 capture card works well for creators who travel or switch gaming setups often. Throw it in your bag and stream or record wherever you are, at home, events, LAN parties, streaming or studio sessions
Hard time limits kill the process
The Tasks guide says a time limit kills the executing process by force. That kill can land in the middle of a file write, so a partial file must never be treated as a finished artifact. Set the subprocess timeout below the queue’s hard limit so your wrapper can clean up and report before the process is killed. This is an engineering choice, not a Celery requirement.
To make redelivery safe, apply these rules in the task itself:
- Write each download under a temporary name inside the job directory, and rename it to the final name only after post-processing and the output check succeed.
- Key the job directory by a stable job ID, so a rerun finds the same location instead of creating a new one.
- At the start of each run, decide explicitly whether to resume, overwrite, or delete the job directory, and encode that decision in the task rather than leaving it to chance.
- Record completion only after the checks from section 3 pass.
5. Long downloads starve short tasks
A media job can hold a worker slot for minutes, and short tasks queued behind it wait. The Celery Optimizing guide describes routing long-running and short-running work to dedicated workers. It also documents two settings that limit how many tasks a worker reserves: task_acks_late and worker_prefetch_multiplier. The guide attaches the same caveat as before: late-acknowledged work must be idempotent. (Celery Optimizing guide)
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →task_acks_late = True
worker_prefetch_multiplier = 1Route the media task to its own queue with your task routing configuration, then run a worker that consumes only that queue:
Best Value
- 4K60 Capture: Record in cinematic quality with crisp detail and vivid colors
- HFR Support: Play and capture in 1440p120 or 1080p240
- HDR10 Support: Capture brilliant HDR content with tone mapping on Windows
- Cross-Platform Compatible: Works with PS5, Xbox Series X/S, Switch 2, and more
- Analog Audio In: Capture in-game chat or commentary with 3.5mm input
celery -A media_app worker -Q media --concurrency=2Size the concurrency to the CPU and disk your FFmpeg merges can use. A value of 2 is an example, not a recommendation for any particular host.
Troubleshooting sequence
Work through the symptoms in the order they appear in a job’s life. The first check in each row is the cheapest way to confirm or rule out the cause.
| Symptom | Check first | Likely cause and fix |
|---|---|---|
| “Not found” error for ffmpeg or ffprobe | The shutil.which output from inside the worker process |
The binary is absent from the worker runtime. Install it and restart the workers. |
| Download finishes, final file missing or incomplete | Stage name and return code in the job record | A post-processor failure hidden by a file-existence check. Treat any non-zero exit as a failed job. |
| A later stage reports the output missing | The absolute path logged after the download, versus the path the next stage reads | A relative path or mismatched directory. Use -P with an absolute directory and an -o template. |
| Duplicate or conflicting files after a retry | Whether late acknowledgement is enabled, and whether the task’s side effects are repeat-safe | A non-idempotent task running under late acknowledgement. Use a job-scoped directory and temporary-name-then-rename. |
| Job disappears or stalls after a worker restart | Acknowledgement mode, and worker logs for killed processes | Early acknowledgement after a crash, or a hard kill that interrupted a write. Confirm the job record exists and clean up partial directories on restart. |
| Short jobs wait behind media jobs | Queue depth per queue, and worker concurrency per queue | Media and short tasks share one worker pool. Route media tasks to a dedicated queue. |
Choosing an output strategy
The three common layouts differ mainly in how they handle reruns and cleanup. The ratings below are design judgments from the documented behavior, not measurements.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Quick Recap
| Strategy | Deterministic path | Collisions on retry | Partial-file cleanup | Intermediates available for diagnosis |
|---|---|---|---|---|
| Shared folder with default names | No; depends on folder contents and template | High; reruns can overwrite or mix with other jobs | Difficult; partial files sit beside completed ones | Hard to attribute to a job |
Per-job directory with %(id)s.%(ext)s |
Yes, when the job ID is stable | Lower; each job is isolated | Simple; remove the job directory | Yes, until cleanup runs |
| Per-job directory with temporary name and rename on success | Yes | Lowest of the three; the final name appears only after success | Simple; temporary names mark incomplete work | Yes, if the directory is kept on failure |
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
