To prevent duplicate webhook work in Node.js, use the provider’s stable event ID as an idempotency key, claim it atomically, and enforce durable uniqueness in PostgreSQL. Put the event claim and database changes in one transaction. Redis can speed up duplicate checks, but it should not be the only record if losing or expiring a key could cause the event to run again.
Why webhook idempotency matters
Webhook senders may deliver an event more than once. For example, your handler can finish its work but the response can be lost or delayed, leaving the sender uncertain whether to retry. Your service therefore needs to distinguish a new event from a repeat delivery, including when two copies arrive at nearly the same time.
As an Amazon Associate I earn from qualifying purchases.
Idempotency means that processing the same event again does not repeat its business effect. It does not mean ignoring every repeated HTTP request blindly: the handler must first establish that the event has already been claimed or completed according to your processing design.
Recommended Free Tools
Choose a stable, properly scoped key
After authenticating the request as the provider requires, extract its stable event or delivery identifier. A timestamp for the current delivery or a locally generated request ID is unsuitable: either can change on retry and make the same logical event appear new.
#1 Best Overall
Scope the key to the event namespace. A composite key such as provider plus event ID may be sufficient; include a tenant or endpoint as well when IDs are only unique within those boundaries. This prevents unrelated event sources from colliding.
Pick the claim architecture
| Design | Strength | Trade-off |
|---|---|---|
| Redis-only atomic claim | A Redis SET with NX can claim a key atomically and an expiry can reclaim it later. Redis describes the NX flag as avoiding a check-then-insert race, even under concurrent requests: Redis: How to do data deduplication with Redis. |
Expiry, eviction, restart, or data loss can remove the only record and allow a repeat to be processed. |
| PostgreSQL unique constraint | A unique or primary-key constraint, combined with INSERT ... ON CONFLICT DO NOTHING, is a durable, concurrency-safe gate. |
Every claim involves the database. Whether a Redis fast path is worth adding depends on traffic, latency needs, and the cost of operating another dependency. |
| Redis fast gate plus PostgreSQL backstop | Redis can filter repeats quickly while PostgreSQL retains durable uniqueness. | The additional layer does not replace the database constraint or the transaction that couples the claim to business changes. |
Do not deduplicate with a separate SELECT followed by an INSERT. Two concurrent handlers can both read “not present” before either inserts. Let an atomic claim operation or the database uniqueness constraint decide which request wins.
Rank #2
Deduplicate webhooks in PostgreSQL from Node.js
For database-backed work, the durable event record and the business changes should commit together. The example below assumes a PostgreSQL client that exposes transaction control, a table with a unique key, and an authenticated event whose provider and ID have already been validated. Adapt the client calls and SQL types to your installed driver and schema.
const client = await pool.connect();
try {
await client.query('BEGIN');
const claim = await client.query(
`INSERT INTO processed_webhook_events (provider, event_id)
VALUES ($1, $2)
ON CONFLICT (provider, event_id) DO NOTHING
RETURNING event_id`,
[provider, event.id]
);
if (claim.rowCount === 0) {
await client.query('COMMIT');
return { duplicate: true };
}
// Apply database-backed business changes using this transaction.
await applyBusinessChanges(client, event);
await client.query('COMMIT');
return { duplicate: false };
} catch (error) {
await client.query('ROLLBACK');
throw error;
} finally {
client.release();
}The table needs a uniqueness rule matching the key scope—for example, a unique constraint on (provider, event_id), or on additional tenant or endpoint columns if required. Only the request whose insert returns a row should apply the database effects. If the insert conflicts, the transaction has established that the key already exists and no business changes should run for that delivery.
Rank #3
Because the claim and database changes are in the same transaction, a failure before commit rolls both back. A later provider retry can then attempt the event again. Once committed, a conflicting delivery is a confirmed duplicate and should receive a successful acknowledgement so the sender does not keep retrying completed work.
Use Redis as an optional fast layer
A Redis claim can reduce repeat work or database load, but its state is temporary compared with a durable PostgreSQL uniqueness constraint. Configure the TTL to cover the provider’s retry horizon plus realistic processing delay; the appropriate duration is provider-specific, not a universal number. Keep PostgreSQL’s unique constraint as the backstop if correctness depends on not applying the database effect twice.
Rank #4
Redis state can disappear through expiry, eviction, restart, or data loss. If Redis is used before the database transaction, treat a positive Redis result as a signal to verify the durable event state where necessary, and do not let Redis absence authorize unguarded business work. The PostgreSQL insert remains the authoritative claim for transactional database processing.
Free tools Windows power users keep installed
One-click scans. No signup required.
Redis’s official deduplication tutorial demonstrates the atomic command pattern, but its example uses Bun and Express; it is useful for understanding the Redis operation, not proof of a single production architecture for every Node.js service.
Best Value
Protect effects outside the database
A PostgreSQL transaction cannot atomically commit an external API call, email, or message-broker publish. If the database commits but the remote call fails—or the remote system acts but its response is lost—the two systems can disagree about what happened.
- Pass a downstream idempotency key when the external service supports one.
- Use state-aware operations, such as upserts or guarded state transitions, where they fit the effect.
- For message publication or other remote work, record enough state to recover and retry deliberately rather than assuming the database transaction covered the remote action.
- Define how to investigate and resolve ambiguous outcomes, especially for payments or other consequential work.
For correctness-critical deduplication, use exact key checks such as Redis SET NX or PostgreSQL uniqueness. A bitset or Bloom-style filter can produce false positives, so it should not be the sole reason to discard an important event.
Set retention around retries and replay
Redis expiration should span the sender’s documented retry window and the processing delays your system reasonably expects. Keep durable PostgreSQL event IDs for the application’s manual replay and recovery window. Provider retry schedules, response deadlines, and replay policies vary, so check the specific provider’s current documentation before choosing retention or acknowledgement behavior.
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Longer retention improves protection against old replays but increases the number of stored identifiers; shorter retention reduces storage but narrows the period in which a repeat is recognized. Choose the horizon based on actual provider behavior and your operational replay policy rather than treating a sample TTL as generally correct.
Operational checks before shipping
- Verify that authentication happens before trusting the event ID.
- Confirm the database uniqueness constraint matches the full event namespace.
- Test simultaneous deliveries of the same event; exactly one should win the database claim.
- Force a business-change failure and confirm the transaction rolls back, allowing a retry to claim the event.
- Confirm a committed duplicate receives the provider-appropriate successful response.
- Exercise Redis expiry or loss and verify PostgreSQL still prevents duplicate database effects.
- Test external-effect failures and ambiguous responses with the downstream idempotency or recovery behavior you have chosen.
Provider-specific retry rules and response deadlines are not interchangeable. Verify them against the provider’s own documentation and the versions of the Node.js SDK and database client deployed in your service; implementation guides and tutorials illustrate patterns but do not establish provider requirements.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
