October DealsAmazon USOctober deal check: compare before you payAmazon US: current deals, useful picks and tech finds.Check DealsWindows FixRecommendedWindows errors stealing your time? Find the fix fastScan stability, cleanup and performance issues.Fix NowOctober DealsAmazon USDeal season is back - check today's better picksAmazon US: current deals, useful picks and tech finds.See Picks×
Skip to content
World desk5 min

The Webhook Bug That Passed Every Test and Code Review

Webhook retries can repeat a business action even when every request has a valid signature. Durable event-ID deduplication and idempotent effects close the gap.
Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

A webhook can pass signature checks, return a successful response, and still trigger the same business action twice. The usual gap is not that every test failed; it is that tests and reviews covered one valid delivery at a time, while real senders retry, redeliver, or deliver concurrently. The fix is to verify the request correctly, deduplicate the stable event ID durably, and make the business effect safe to repeat.

The title describes a general failure pattern, not a documented incident at a named company. Provider retry rules and signature formats differ, so apply the sending service’s current documentation to your integration.

Why can a valid webhook run twice?

Signature verification establishes that a request’s signed content is authentic and has not been altered in transit. It does not prove that the request is the first delivery of an event. A legitimate retry can carry a valid signature, and a previously valid request may also be replayed if the implementation has no freshness or deduplication controls.

Consider a sender delivering an event that asks your application to capture a payment or send a notification. The receiver verifies the signature and commits the action, but its response is delayed or lost. The sender cannot know whether the work succeeded, so it tries again. If the receiver has no durable record of the event and the action itself is not idempotent, the second valid request repeats the effect.

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Concurrent deliveries create another failure mode: two handlers can both check that an event has not been seen, then both proceed before either records it. A check followed by a separate insert is not safe unless the storage operation makes the claim atomic.

What should signature verification prove?

Verify the exact raw request body

Calculate the expected signature from the provider’s specified inputs, including the unmodified request bytes where required. Verify it before parsing or rewriting the payload and before performing business work. Middleware that parses and transforms a body can change the bytes used for verification. GitHub warns that changing the payload or headers can cause verification failures, and Shopify’s webhook documentation specifically warns that body-parser middleware can alter the input needed for HMAC verification.

Rank #2
Sale
HTML and CSS: Design and Build Websites
  • HTML CSS Design and Build Web Sites
  • Comes with secure packaging
  • It can be a gift option

Compare signatures safely

Use a constant-time comparison for the calculated and supplied signatures rather than ordinary string equality. GitHub’s guidance on validating webhook deliveries covers HMAC validation and constant-time comparison. Keep the signing secret private, and follow the provider’s exact signature format and key-rotation instructions.

Use freshness and event identity for different jobs

Where the provider signs an attempt timestamp or specifies a freshness rule, reject requests outside the permitted window. This limits stale replay, but it does not replace deduplication: a retry may be fresh while still representing the same event. A stable event ID identifies the event across attempts; an attempt timestamp describes when that particular delivery was made. The Standard Webhooks specification distinguishes signing metadata, attempt timestamps, and stable event IDs.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

How do you make processing idempotent?

  1. Authenticate first. Preserve the raw body and verify its signature and any required freshness metadata before trusting the payload or starting work.
  2. Claim the stable event ID atomically. Store it durably with a uniqueness constraint or equivalent atomic operation. If another request already claimed the ID, do not start the same business operation again.
  3. Make the claim and effect crash-safe. A process can stop after recording an event but before completing its work, or after completing the work but before recording success. Use a transaction when the record and effect share a transactional store. For work crossing system boundaries, use a durable inbox/outbox or an equivalent recovery pattern that tracks pending and completed work.
  4. Make downstream effects repeat-safe. Pass an idempotency key to a downstream API when supported, or design the operation so a repeated request converges on the intended state instead of creating another charge, row, or notification.
  5. Acknowledge known duplicates appropriately. For an event already completed, skip its effects and return the provider-appropriate success response so a harmless duplicate does not provoke more retries. Keep incomplete or failed work recoverable rather than marking it complete prematurely.

A queue can make processing asynchronous, but a queue by itself does not guarantee exactly-once business effects. The consumer still needs durable deduplication and recovery behavior for duplicate messages, crashes, and retries.

How should the receiver handle timeouts and retries?

The sender retries when it does not receive an acknowledgement it considers successful; the precise retry cadence, response codes, and redelivery semantics vary by provider. GitHub Docs says: “Your server should respond with a 2XX response within 10 seconds of receiving a webhook delivery.” Its webhook best practices describe queueing as a way to keep acknowledgement fast while processing work asynchronously.

Rank #4
Sale
Web Design with HTML, CSS, JavaScript and jQuery Set
  • Brand: Wiley
  • Set of 2 Volumes
  • A handy two-book set that uniquely combines related technologies Highly visual format and accessible language makes these books highly effective learning tools Perfect for beginning web designers and front-end developers

Where a durable queue is appropriate, validate the request and safely record or enqueue the work before acknowledging it. If the receiver acknowledges before the work is durable, a crash can lose the event; if it waits for slow business work, a timeout can invite another delivery. Design the acknowledgement boundary around what the system can reliably recover.

Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

What tests catch this failure pattern?

Test sequences and failure points, not just whether one request returns HTTP 2XX. OWASP’s draft Webhook Security Guidelines includes checks for invalid or missing signatures, replay, duplicate event IDs, and oversized payloads. The draft can change; use it as guidance alongside the sender’s current requirements.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
  • Deliver the same valid event more than once; assert one business effect and the correct final state.
  • Send two copies concurrently; assert that only one atomic claim succeeds and downstream effects are not duplicated.
  • Test a valid replay within the freshness window and a stale request outside it.
  • Send an invalid signature for a body that otherwise has a previously seen event ID; confirm authentication still fails rather than allowing deduplication to bypass signature checks.
  • Simulate a response lost after the effect commits, then retry the delivery.
  • Force partial failure between claiming an event and completing its work; verify that retry or recovery finishes safely without repeating completed effects.
  • Restart the process between attempts and confirm deduplication survives in durable storage.
  • Test malformed, missing, and oversized inputs according to the provider’s limits and your security policy.

For each scenario, assert the number of committed business effects and the resulting state, not only the handler’s HTTP status. This exposes cases where the transport response looks correct while the application does the wrong thing.

What belongs in a webhook code review?

  • Does verification use the exact raw bytes and the provider’s current signature format?
  • Is signature comparison constant-time, and is freshness checked where the provider supports it?
  • Is a stable event ID claimed in durable storage with an atomic uniqueness guarantee?
  • Can concurrent deliveries, crashes, or retries repeat a side effect or strand work?
  • Does the handler acknowledge only after it has made the event safely recoverable, and within the sender’s deadline?
  • Do tests cover duplicate, concurrent, replayed, and partially completed attempts as well as ordinary valid delivery?

These checks target the gap between “this request is authentic” and “this event’s business effect should happen only once.”

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Leave a Reply

Your email address will not be published. Required fields are marked *

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

More from the Wire

  1. World desk4 min
    How to Spot an AI Voice Scam Before Sending MoneyDon’t rely on how a caller sounds. Pause, call back through a known number, and verify the emergency with another trusted person before sending money.
  2. Mountain View desk4 min
    Google’s SynthID Detector: How to Check AI-Generated Images, Video and AudioGoogle’s SynthID Detector looks for an embedded watermark in supported images, video and audio. Here is what its results do—and do not—show.
  3. Redmond desk20 min
    How to create a link to File or Folder in Windows 11Windows 11 gives you several ways to point to a file or folder without moving or duplicating it. You can create a desktop shortcut,…
Recommended PC Tool
Recommended PC Tool
PC Slower Than It Used to Be?Free scan - under a minute
Outdated Drivers Are Slowing You DownFree scan - exact matches

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.