The Tool Desk
Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →When Redis is full and BullMQ cannot add a job, that job may not be recorded anywhere: if the application’s enqueue call fails and no independent store captures the work, it is lost from the queue’s perspective. Matheus Morett says that happened in production in April 2025, when incoming messages outpaced processing and lost negotiation messages followed. His response, bullmq-outbox, saves failed enqueues outside the queue’s Redis and retries them later. It can turn a write failure into delayed processing—but only when the fallback store and recovery worker remain available.
What failed when the queue fell behind
In Morett’s account, a production workload grew more slowly than messages arrived. Waiting jobs accumulated in Redis, which reached its capacity; BullMQ then failed to enqueue new jobs. Because the queue was the only record of those new messages, rejected enqueues became lost negotiation messages. This is the author’s reported incident, not an independently audited case study or a performance benchmark. Morett, who identifies himself as Monest’s CTO, summarized the constraint as: “Redis is the ceiling.” Morett’s account and implementation
As an Amazon Associate I earn from qualifying purchases.
The important distinction is between a job already stored in Redis and an enqueue attempt Redis rejects before storing it. Redis memory pressure can affect existing queue state, but retention settings for completed or failed jobs do not rescue a new job that was never accepted.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Scan for outdated or missing drivers - takes under a minute3Clear out junk files and repair common Windows errorsCan Redis Cluster make one hot BullMQ queue bigger?
Redis Cluster can distribute a dataset across nodes, but it does not split an individual hash slot among them. Redis Cluster has 16,384 hash slots; keys map to slots, and nodes own subsets. Multi-key commands, transactions, and Lua scripts require the keys they touch to share a slot. Hash tags—text inside braces, such as {orders}—can deliberately put related keys in the same slot. Redis Cluster specification
#1 Best Overall
BullMQ operations use multiple related keys for a queue. As Morett explains, those keys must be colocated for scripts that operate on them. Adding cluster nodes can increase total cluster capacity and spread different queue slots across nodes; it cannot divide the slot holding one queue’s related keys across several nodes. This matters when one queue is the bottleneck, but it does not make clustering pointless.
Before treating Redis capacity as the cause, distinguish a hot queue or slot from broader cluster pressure, a slow consumer, and memory retained by finalized jobs. BullMQ’s guide says completed and failed jobs are retained by default in dedicated sets. Count- and age-based auto-removal can limit that retention, but removal is lazy. BullMQ’s auto-removal guide
Rank #2
Unique job IDs can help suppress duplicate additions while the original job remains in the queue. If that job has been removed, its ID no longer prevents a later add with the same ID. Neither auto-removal nor a unique ID preserves an enqueue Redis rejected before storing.
How the outbox changes the failure
Morett’s first version wrapped queue add() calls. When an add failed, it wrote the queue name, job name, payload, options, and a PENDING status to DynamoDB. A cron job ran every 15 minutes and tried to enqueue pending records through the real BullMQ queue. The original enqueue error was still rethrown to the caller, so the application could decide what to tell the user instead of silently treating the job as queued.
The package described in Morett’s article generalizes the pattern with createOutbox({ store }), wrapQueue(queue), and flush(limit). Its storage interface is represented by save, loadPending, markProcessed, and markFailed. The author says the package ships without storage adapters and provides example implementations to adapt for Postgres, Redis, MongoDB, or DynamoDB. These are design details from the article; the package’s current release state and compatibility were not independently verified.
Morett describes wrapQueue as a Proxy and the package as structurally typed, with no direct BullMQ dependency, to pass through queue methods and support BullMQ v5, v6, and Pro. Treat that as the author’s design description, not a guarantee of present-day compatibility.
What a safe replay needs
Replaying a job is not simply retrying an add(). It must preserve enough of the original request to behave as intended, and the recovery path must be able to run while the primary queue is unavailable.
- Keep identity and retry behavior. Morett specifically calls out preserving
jobId, attempts, and backoff options. Stable IDs can help make re-enqueue idempotent while the corresponding job still exists, but do not establish exactly-once execution of downstream side effects. - Keep failure visible. Rethrowing the original enqueue error lets the application distinguish “accepted by the queue” from “saved for later recovery.” The caller’s response should reflect that distinction.
- Separate the failure domains. Morett’s original recovery scheduler used a dedicated Redis. If the scheduler depends on the same Redis failure it is meant to recover from, it may not run. The fallback store and the process that reads it must also survive the primary queue’s failure.
- Make replay safe downstream. A recovered job may be attempted after an earlier attempt’s outcome is uncertain. Consumers should make side effects idempotent or otherwise handle duplicates; an outbox by itself cannot promise exactly-once processing.
- Watch delay, not only volume. Morett exposes
ageMsthroughonJobRequeued. The age of the oldest pending or replayed work indicates how long users have been waiting. A replay count alone cannot show whether recovery is timely.
When buffering is—and is not—a good fit
An outbox is useful when preserving a job is more important than completing it immediately. But it changes a hard failure into deferred work only if the fallback store is writable and durable, records are retained, recovery runs, and downstream processing tolerates replay.
Best Value
Morett excludes real-time conversation queues from his use case: a replay after the original 15-minute interval could be worse than dropping a stale interaction. For such work, the system needs a product-specific policy for stale jobs rather than automatic buffering.
His operational advice is to configure reserved memory for the Redis service so memory exhaustion produces a catchable error rather than a stalled connection. He also says the package’s integration tests use a real Redis configured near its memory limit. Those are the author’s reported practices, not independently reproduced results. Morett’s account and implementation
What to monitor during recovery
Alert on the age of the oldest pending outbox record and the duration of recovery delays, alongside queue depth and Redis health. An outbox that is accumulating old jobs is not healthy merely because each flush reports some successful requeues. Morett’s article includes an example flush output of 12 requeued, 0 failed, and 3 skipped; those are example values, not general recovery rates.
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
