Recommended Free Tools
To make a delivery happen once when your Node.js worker retries, make the delivery operation idempotent. Give each logical delivery a stable identity, and enforce that identity at the point where the side effect is committed, so replaying the same job cannot create a second delivery. Queue retries and queue-level deduplication help control repeated jobs, but neither makes an external side effect exactly once on its own.
Why a retry can deliver twice
A worker can fail after it has already done the work it was asked to do. The common sequence looks like this: the processor sends the email, writes the row, or calls the payment API; the process then crashes, loses its connection, or times out before the job is marked complete. The queue sees an unfinished job and runs it again. From the queue’s point of view nothing went wrong. From the customer’s point of view, the delivery happened twice.
Two platform behaviours make this the normal case rather than a rare edge. BullMQ retries a job after a processor failure when retries are configured. Amazon SQS standard queues are documented as at-least-once: AWS states that a message may be received again in rare cases, and advises designing consumers to be idempotent. Your code therefore has to assume that the same work can arrive more than once.
Retries and idempotence solve different problems
A retry policy decides when a failed job is attempted again. It says nothing about whether repeating the side effect is safe. BullMQ’s retry guide covers attempts and fixed or exponential backoff; those settings control timing and count, not correctness.
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstall#1 Best Overall
Idempotence is the property that matters for correctness. BullMQ’s idempotent-jobs guidance defines it by the final system state: a job is idempotent when the end state is the same whether it succeeded on the first attempt or only after one or more retries. That is the target. A delivery is safe to retry when running it again leaves the system in the same state as running it once.
- Retries answer: how many more attempts, and how far apart?
- Idempotence answers: what happens to the world if an attempt repeats?
- A higher retry count without idempotence increases the chance of duplicate side effects; it does not reduce it.
What queue deduplication does and does not cover
BullMQ offers job deduplication tied to job state or to a time-to-live (TTL). It can stop a repeated addition from creating another job while the matching job is still in the relevant state, or within the configured window. That is useful admission control for the queue. It is not evidence that the email was sent once.
BullMQ’s throttle-jobs guidance also warns about retention. If a completed or failed job that used a reused job ID has been removed, it no longer counts as an existing duplicate, so a later add with the same ID can create a new job. Check your removal settings before relying on job IDs as a long-term guard.
Amazon SQS FIFO queues suppress duplicate sends within a five-minute deduplication interval when you use content-based deduplication or supply an explicit deduplication ID. This is a send-side behaviour with a stated window. Outside that window, or once a message has been received and processed by a consumer that then fails before recording completion, the queue does not protect your side effect. The AWS exactly-once processing page describes this scope; read it as a statement about sends in that window, not as an end-to-end guarantee.
Rank #2
Choosing where the idempotency boundary sits
The right boundary depends on where the side effect lands. The table below compares the layers that are commonly used, and what each one actually protects.
| Layer | What the documentation establishes | What it protects, and what it does not |
|---|---|---|
| Application-level idempotence | BullMQ defines idempotent jobs by unchanged final state after a retry. | Protects the intended operation. Your code must define the logical identity and enforce it. |
| BullMQ job ID or deduplication | Repeated adds can be ignored while a matching job exists, or according to the configured deduplication mode or TTL. Removal affects later duplicate detection. | Controls queue admission. Does not make a third-party side effect idempotent. |
| Amazon SQS standard queue | AWS documents that a message may be delivered again in rare cases and advises idempotent consumers. | The consumer must tolerate repeated processing. The queue does not supply that tolerance. |
| Amazon SQS FIFO queue | Duplicate sends are suppressed within the five-minute deduplication interval when content-based or explicit deduplication IDs are used. | Scoped to sends in that interval. Not a universal exactly-once effect. |
Sources for the table: BullMQ: Idempotent jobs, BullMQ: Deduplication, BullMQ: Throttle jobs, Amazon SQS: At-least-once delivery, and Amazon SQS: Exactly-once processing.
In practice, the layers stack. Queue deduplication reduces accidental duplicate jobs. The idempotent write at the side-effect boundary is what stops a repeated attempt from producing a second logical delivery.
Building a logical delivery key
Everything depends on a key that identifies the logical delivery, not the attempt. The rule is simple to state:
Rank #3
- A retry of the same business event reuses the same key.
- A genuinely new delivery, such as a second notice for a new order, gets a different key.
Derive the key from the business event or request identity, such as an order ID plus a notification type, or a client-supplied request ID. Do not derive it from the BullMQ job ID, a timestamp, or a random value generated inside the processor, because those change between attempts or between enqueues. This is an implementation pattern consistent with how BullMQ and AWS describe deduplication and idempotency keys; neither source prescribes a particular key format.
Enforcing the key in your database
For a delivery you record in your own database, a uniqueness constraint is the most direct enforcement. Concurrent attempts can both start, but only one insert can succeed. An illustrative PostgreSQL table and a Node.js claim step look like this:
CREATE TABLE deliveries (delivery_key text PRIMARY KEY, status text NOT NULL, created_at timestamptz NOT NULL DEFAULT now());
const { rowCount } = await pool.query(
`INSERT INTO deliveries (delivery_key, status)
VALUES ($1, 'pending')
ON CONFLICT (delivery_key) DO NOTHING`,
[deliveryKey]
);
if (rowCount === 0) {
// Another attempt already claimed this delivery. Read its status and return.
return;
}
// Only the attempt that inserted the row performs the side effect.
await sendNotification(payload);
await pool.query(
`UPDATE deliveries SET status = 'sent' WHERE delivery_key = $1`,
[deliveryKey]
);This sketch has a gap you must close. If the worker crashes after the insert but before the side effect completes, the row stays pending forever. Add a lease or an updated-at timestamp, and let a later attempt take over a pending row only after the lease expires, then check the side-effect system for evidence that the send happened before repeating it. Where the side effect cannot be checked, you have to accept an at-most-once or manual-review path for that narrow window.
Rank #4
Calling an external API
When the side effect is an external API call, your database key is not enough on its own. If the provider documents an idempotency mechanism, pass your logical key through it so the provider can recognise repeats. If it does not, you cannot assume repeated calls are safe, and you need to design around that: record intent before the call, reconcile against the provider’s records after a crash, or choose a flow the provider can verify. Check the provider’s current documentation for the specific mechanism, parameter names, and retention period. Do not assume one exists because the pattern is common.
Keeping the worker step small and retries bounded
BullMQ’s idempotent-jobs guidance recommends simple, atomic jobs. A job that performs many actions can make partial progress, and then rollback and tracking become difficult. Splitting the work so that each job has one side effect, and so that the idempotent claim happens first, makes the failure window smaller and easier to reason about.
For transient failures, configure a bounded number of attempts with fixed or exponential backoff. Then watch the failures that exhaust those attempts; they need a human or a dead-letter path, not an unlimited loop. Configure retries as a cost and latency control, and treat idempotence as the correctness control.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Testing the failure window
The failure that matters is a crash after the side effect has been committed and before the job is recorded as complete. You can reproduce it deliberately in a staging environment:
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Repair Windows errors before they cause bigger problems3Scan for outdated or missing drivers - takes under a minute- Enqueue one job for a single logical delivery key.
- In the processor, let the claim and the side effect succeed, then terminate the worker process before the job completes, for example with
process.exit(1)placed after the side effect. - Restart the worker so that the queue retries the job with the same logical key.
- Count the logical deliveries in your own store and in the downstream system. Confirm there is exactly one.
- Repeat the scenario for a concurrent retry, starting two workers against the same job, and confirm the claim still yields one delivery.
Run the same steps after any change to the claim logic, the retention settings for job IDs, or the retry configuration.
Version and scope limits
The behaviours described here come from BullMQ’s and AWS’s published documentation as of October 2026. Option names, retention defaults, and deduplication modes can change between releases. Before you ship version-sensitive code, read the documentation for the BullMQ version you have installed and the current SQS guide. This article does not report benchmarks or measurements of its own; the examples show the pattern, and you should test them against your own stack.
BullMQ’s documentation is at docs.bullmq.io. The retry options are described in Retrying failing jobs, and the idempotence guidance in Idempotent jobs.
The design rule is short: one logical key, enforced where the effect is committed, with queue features used only to reduce accidental duplicates. Build that, and a retry becomes a harmless replay rather than a second delivery.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
