What Is a Webhook? How Webhooks Work and Six Ways They Break in Production

A webhook is an HTTP event notification sent by one service to an endpoint you configure. When a subscribed event occurs, the service sends a request; your application verifies and acknowledges it, then processes the event. The pattern is straightforward, but delivery timing, retries, signatures, and ordering vary by provider—details that matter when a webhook is part of a production system.

How a webhook works

A webhook reverses the usual polling pattern. Instead of your application repeatedly asking a service whether anything changed, the service sends a request when an event occurs. For example, an online store can notify an order-processing application when an order is created.

#	Preview	Product	Price
1		APIs and Webhooks for Beginners: Connect Apps, Automate Tasks, and Build Useful Integrations	$2.99	Buy on Amazon
2		Shelly Pro 3EM 3CT 63 Wi-Fi & LAN 3-Phase Smart Energy Meter	$150.99	Buy on Amazon

As an Amazon Associate I earn from qualifying purchases.

For HTTPS deliveries, Shopify documents a common form: an HTTP POST with a JSON body and metadata in headers. Other providers may use different payload formats, headers, transports, delivery identifiers, and retry policies. A webhook is a common integration pattern, not one universal protocol.

1. Configure a destination and subscription. The receiving application provides an endpoint, and the sending service is configured to deliver selected event types there.
2. The event occurs. The sender constructs a delivery containing the event data and, depending on the provider, identifiers, version information, and a signature.
3. Your endpoint receives and verifies it. Preserve the request body in the form required for signature verification, check the signature, and validate the event type and action before trusting the data.
4. Persist or enqueue the event. Record it durably so a process crash after the HTTP response does not silently lose accepted work.
5. Acknowledge promptly, then process safely. Respond within the provider’s documented time limit and handle the event with duplicate and ordering safeguards.

Keep the provider’s delivery documentation close to the implementation. Do not infer a retry schedule, delivery guarantee, or ordering rule from another service’s behavior.

#1 Best Overall

Sale

APIs and Webhooks for Beginners: Connect Apps, Automate Tasks, and Build Useful Integrations

$2.99

Buy on Amazon

Why webhooks break in production

These are six common failure classes, not a promise that every provider retries, orders, or reports deliveries in the same way.

1. The endpoint cannot be reached

DNS errors, network routes, firewall rules, a stopped listener, or incorrect host configuration can prevent a sender from connecting. GitHub’s troubleshooting guidance distinguishes connection errors and recommends checking network access and current GitHub IP information. If you use IP allowlists, treat them as operational data that may need updating rather than as a set-and-forget configuration. See GitHub’s webhook troubleshooting guide.

2. The receiver takes too long

A handler that performs lengthy business logic before responding can exceed the sender’s deadline. GitHub recommends a 2XX response within 10 seconds; it terminates slower deliveries and marks them failed. Queue the work asynchronously so the request handler can verify and durably accept the event, then respond promptly. The queue must itself be durable: acknowledging before saving the event can lose it if the application crashes. See GitHub’s webhook best practices.

3. The response is rejected

A non-2XX status or an invalid HTTP response may be treated as a failed delivery. Log the status and consult the sender’s delivery record to distinguish an application rejection from a connectivity problem. Return success only after the event has been safely accepted or persisted—not merely because the request reached your handler.

4. Signature verification fails

Common causes include using the wrong secret, checking the wrong header or algorithm, or allowing middleware to alter the request body before verification. Shopify calculates its HTTPS HMAC from the raw request body; parsing JSON and serializing it again can change the bytes and invalidate the check. GitHub recommends the X-Hub-Signature-256 header with HMAC-SHA256. Verify using the provider’s documented input and method before acting on the payload. See Shopify’s verification guide and GitHub’s best practices.

5. A delivery is repeated

Retries and ambiguous network failures can result in the same event being handled more than once. Shopify explicitly notes that duplicate deliveries can occur and provides delivery identifiers. Record an appropriate provider event or delivery ID, and make consequential business actions idempotent so repeating a delivery does not repeat a charge, shipment, or other side effect.

An Idempotency-Key is one possible pattern only when the receiving server supports and documents it. MDN labels the header experimental and non-standard; do not assume a webhook sender or receiver implements it. See MDN’s Idempotency-Key reference.

Rank #2

Shelly Pro 3EM 3CT 63 Wi-Fi & LAN 3-Phase Smart Energy Meter

• The Shelly Pro 3EM 3CT 63 is a next-gen DIN rail-mountable energy meter for single or three-phase installations, featuring a 63A, 3-phase current transformer for non-contact measurements. It supports 4-quadrant measurement, optical pulse indication of energy usage, and is photovoltaic-ready. *It doesn't have a built-in relay; contactor control requires a Shelly Pro Addon attached to the device.
• Professional Smart Meter - Shelly Pro 3EM-3CT63 is a professional smart meter that reports accumulated energy, voltage, current, active, and apparent power per phase in real time. It stores data for up to 60 days in 1-minute intervals and includes a real-time clock to maintain accurate time if the SNTP server connection is lost.
• Ideal for business energy measurement - In commercial buildings, it helps monitor energy usage across floors or departments allowing accurate cost allocation and identification of energy wastage. In manufacturing plants it tracks energy consumption of heavy machinery, optimizing usage to reduce operational costs. For store owners it monitors energy usage of systems like lighting, HVAC § refrigeration, helping to identify inefficiencies § reduce energy bills while supporting sustainable practices
• Shelly Customer Service - Shelly is one of the fastest-growing Smart Home brands in the world with devices, providing solutions for the automation of private homes, buildings and businesses. We provide our customers with professional support and a 5 years device warranty.
• Shelly Smart Control App will help you control your Shelly devices remotely and will send notifications for all automated events in your home. You can easily configure devices and manage their settings individually, or you can create personalized scenes by combining Shelly devices to trigger certain actions in your home automation.

$150.99

Buy on Amazon

6. Events arrive late or out of order

GitHub notes that deliveries may arrive in a different order from the events and can take minutes to appear. If each notification overwrites state blindly, an older event can undo a newer one. Use provider timestamps and identifiers where available, avoid applying stale state, and reconcile with the provider’s authoritative API when correct ordering matters. See GitHub’s troubleshooting guide.

Build a receiver that is secure and recoverable

Protect the endpoint and secret

• Use HTTPS and keep certificate verification enabled.
• Store a high-entropy signing secret securely; do not put credentials in the payload URL.
• Verify the signature over exactly the input required by the provider. For Shopify HTTPS deliveries, that means the raw body and the app client secret.
• Do not trust a payload or perform its requested action until signature verification succeeds.

Accept only events your application understands

• Subscribe only to event types the application needs.
• Check both event type and action before acting. Providers can add event types and actions over time.
• Handle unknown events safely—for example, record the delivery and decline to run business logic—rather than assuming every payload matches an older schema.

Acknowledge only after durable acceptance

Keep the HTTP request path short: verify, validate, persist or enqueue, and respond. Move slow work into a recoverable background process. A fast response without durable acceptance is not reliability; it can make a lost event harder to detect because the sender believes delivery succeeded.

Make retries safe and observable

Track event and delivery identifiers, timestamps, event type, signature result, response status, and processing state. Avoid logging secrets or unnecessary sensitive payload data. Use provider IDs where they identify repeats, and add application-level idempotency for consequential side effects. A generic HTTP idempotency header is not a substitute for a documented contract between sender and receiver.

Compare the provider’s actual delivery contract

Before integrating a webhook, check the relevant provider documentation for each of these behaviors. Shopify and GitHub illustrate why one provider’s rules should not be generalized to another.

Provider behavior can change. For Shopify, use the delivery structure, verification, troubleshooting, and current Webhooks API reference for the relevant configuration. The documented Shopify retry count and window are Shopify-specific, not an industry standard.

Investigate a failed delivery

1. Find the sender’s delivery record. Determine whether it attempted the request, what response it observed, and whether the issue is connection, timeout, response status, or verification.
2. Check reachability. Confirm DNS, firewall and allowlist rules, TLS configuration, host availability, and that the expected route is listening.
3. Compare elapsed time and response. Inspect application and proxy logs for handler duration, status code, and malformed responses. For GitHub, compare the delivery with its 10-second response recommendation.
4. Check signature inputs. Confirm the secret, header, algorithm, and exact request bytes match the provider’s instructions. Ensure middleware has not parsed and changed the body first.
5. Trace persistence and processing separately. Establish whether the event was durably accepted, queued, and processed; a successful HTTP response does not prove downstream work completed.
6. Look for duplicate or stale events. Compare event and delivery identifiers and timestamps before replaying or applying a change.
7. Recover deliberately. Use the provider’s redelivery or import procedure, and ensure replaying the event cannot repeat business side effects.

For HTTP responses such as 429 or 503, Retry-After can communicate how long a user agent should wait before another request. It is a response mechanism, not a guarantee that a webhook sender will honor the delay; check the sender’s documentation. See MDN’s Retry-After reference.

Plan for recovery, not just successful delivery

Webhooks are useful for timely event notifications, but a robust integration should not depend on a notification being the only way to discover a change. Keep enough identifiers and processing state to investigate failures, make redelivery safe, and reconcile with an authoritative API or import process when events were missed. GitHub recommends redelivery of missed deliveries; Shopify’s troubleshooting guidance describes recovering missing data after extended downtime by re-subscribing where applicable and importing data.

Quick Recap

SaleBestseller No. 1

APIs and Webhooks for Beginners: Connect Apps, Automate Tasks, and Build Useful Integrations

$2.99

Bestseller No. 2

Shelly Pro 3EM 3CT 63 Wi-Fi & LAN 3-Phase Smart Energy Meter

$150.99

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.