Shadow’s MiniMax Direct pipeline, as described in a self-published DEV Community post by Biffer Rowley, uses PostgreSQL as a durable work queue. Workers claim rows, a listener relays PostgreSQL notifications, and browsers get stage updates over Server-Sent Events (SSE). The design is plausible and common in shape. But the article’s headline claims are not independently verified, and the code it shows doesn’t match its title.
Three things are worth knowing before you borrow from it. The claim code uses FOR UPDATE SKIP LOCKED, not visible advisory locks. A second post under the same byline says the team deliberately avoids LISTEN/NOTIFY. And no memory profile backs the “zero-idle-RAM” framing. The sections below separate what the source says, what the primitives do, and what you would need to measure yourself.
As an Amazon Associate I earn from qualifying purchases.
What the source describes
The central article lays out a six-stage flow for generating media. Stage names and the overall shape below are the author’s account. Nothing in the available material confirms that this is running in production.
Recommended Free Tools
| Stage | What the article says it does |
|---|---|
| 1 | MiniMax Direct text synthesis |
| 2 | Image synthesis |
| 3 | Likeness verification |
| 4 | Hailuo H3 video synthesis |
| 5 | Colour verification |
| 6 | Distribution |
PostgreSQL holds the queue and the per-stage state. Workers claim ready rows. A separate listener process subscribes to PostgreSQL notifications and pushes stage updates to browsers over SSE. The article includes TypeScript examples and reports an 8–14 ms interval from a worker’s commit to the browser painting the update.
#1 Best Overall
Advisory locks or row locks? The code shows row locks
The title leads with advisory locks. The claim method in the article’s code does something different. It opens a transaction, selects a queued row with FOR UPDATE SKIP LOCKED, sets the stage to running, and commits. No advisory-lock function call appears in the shown method. So the example demonstrates row-level claiming, not advisory-lock coordination. Whether advisory locks appear elsewhere in Shadow is not established by the material available.
The distinction matters because the two mechanisms solve different problems.
Rank #2
| Mechanism | What it locks | Typical queue use | Main hazard |
|---|---|---|---|
SELECT … FOR UPDATE SKIP LOCKED |
Rows the query returns; rows already locked by others are skipped | Several workers each take a different ready row without blocking one another | If the worker dies after commit, the row is stuck in “running” unless you add a timeout or heartbeat |
Advisory locks (pg_try_advisory_lock, pg_try_advisory_xact_lock) |
An application-chosen number, not a row | Ensuring only one process runs a job, a singleton listener, or a per-job-ID critical section | Session-level locks belong to the connection, so they can leak or misbehave behind a transaction-mode connection pooler |
LISTEN / NOTIFY |
Nothing; it is a signalling channel | Waking a listener when state changes | Notifications are not stored for listeners that are disconnected, so they can’t be the only record of an event |
These behaviours come from standard PostgreSQL semantics, not from the Shadow article. A claim that commits “running” and then crashes is a general property of the pattern, not a criticism of this specific code.
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallWhere advisory locks would genuinely fit
If you want advisory locks in a design like this, the natural uses are narrow. One is guaranteeing a single active listener or relay process. Another is serialising work on a single job ID, so two stages for the same asset never run concurrently. A transaction-scoped lock (pg_try_advisory_xact_lock) releases at commit or rollback, which avoids the pooler problem of session-level locks. The claim itself is better served by SKIP LOCKED, which is what the article’s own code uses.
The notification-to-SSE relay
The article’s listener subscribes to PostgreSQL notifications and fans them out to browser connections. It gives NOTIFY shadow_stage_done, $1 as a parameterized example. Treat that line as illustrative and not as verified working code. NOTIFY is a utility command, and its payload is normally a literal. When the payload is computed, PostgreSQL provides pg_notify(channel, payload) for exactly that case. A reproducible version would look more like this (my own sketch, untested against Shadow’s schema):
-- worker, same transaction as the state change
UPDATE stages SET status = 'done' WHERE id = $1;
SELECT pg_notify('stage_done', $1::text);
COMMIT;Notifications are delivered only when the sending transaction commits, which is the property that makes “commit, then update the browser” a coherent sequence. Three limits shape how you should build around it:
- Payloads are small. PostgreSQL caps them at just under 8,000 bytes. Send an ID and re-read the row; don’t send the data.
- Delivery is not durable. A listener that is disconnected when the notification fires never receives it. The database table must remain the source of truth.
- A listener needs a dedicated, long-lived connection. That connection can’t sit behind a transaction-pooling proxy and keep its subscription.
Making SSE recoverable
SSE is one-way HTTP streaming (Content-Type: text/event-stream). Browsers’ EventSource reconnects automatically and sends a Last-Event-ID header if the server supplied an id: field. That pairs well with a durable queue. Give each stage event a monotonically increasing ID from a table or sequence. On reconnect, replay events newer than that ID from the table, then resume live relaying. This turns “notifications can be lost” into “notifications are a latency optimisation” and is the main thing the article’s description doesn’t show.
Two operational details tend to bite: reverse proxies that buffer responses and delay events, and the browser limit of about six concurrent HTTP/1.1 connections per origin, which limits multiple open tabs unless the site serves HTTP/2.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.The 8–14 ms figure
The article attributes 8 to 14 milliseconds to the interval between a worker commit and the browser painting, on what it calls a “healthy cluster.” The available excerpt gives no measurement method, workload, sample size, hardware, or network path. It also doesn’t say how a browser paint time was captured or whether clocks were synchronised between server and client. Read it as the author’s observation under unstated conditions, not as what LISTEN/NOTIFY plus SSE will deliver for you. Nothing in PostgreSQL’s documentation, as far as the material available shows, endorses this number or the architecture.
“Zero-idle-RAM”: not established
The title’s memory claim is not supported by any memory profile, process count, or idle-footprint measurement in the available material. A design with a Postgres queue, workers, and one listener process still has resident processes: PostgreSQL itself, the listener, the SSE-serving process, and any worker waiting on a connection. “Idle” can reasonably mean no polling loop burning CPU, which notification-based wake-ups do avoid. But it doesn’t mean no memory is held. If memory behaviour matters to you, measure resident memory per process at zero load and at your expected number of open SSE connections.
The same author’s other post says the opposite
A related post under the same byline, Biffer Rowley, says “We deliberately do not use LISTEN/NOTIFY.” It describes 50 ms polling and publishes a benchmark table comparing the two approaches. That contradicts the central article’s listener. The available material can’t say which account reflects the current deployment, whether the design changed, or whether one post describes a different component. Both are self-published, and the benchmark table is no firmer than the 8–14 ms claim. Don’t average the two accounts into one architecture, and don’t use either set of numbers to pick a winner.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Scan for outdated or missing drivers - takes under a minute3Repair Windows errors before they cause bigger problemsHow to decide for your own system
- Start with the table as source of truth. Claim with
FOR UPDATE SKIP LOCKED, and add a lease column or heartbeat so crashed workers’ rows return to the queue. - Add advisory locks only for a named invariant such as “one relay at a time” or “one running stage per job,” and prefer transaction-scoped locks if you use a pooler.
- Choose the wake-up mechanism by your latency budget. If a 50 ms or longer delay is acceptable, polling is simpler and has no listener connection to babysit. If you need near-immediate updates, add
pg_notifyas a hint and keep a slow poll as a safety net. - Give SSE events IDs and replay from the table so reconnects and missed notifications don’t lose state.
- Measure end to end. For a fair comparison of polling and notification, use the same workload, worker count, database setup, and durability settings. Record commit time on the server and render time in the browser, and report percentiles with sample sizes, not a single range.
The architecture’s skeleton is sound: durable state in PostgreSQL, claims by SKIP LOCKED, signals by notification, and SSE to the browser. The specific claims layered on top of it (advisory-lock coordination, 8–14 ms latency, zero idle RAM, and the current choice between polling and notifications) are unconfirmed, and the author’s own posts disagree on one of them.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
