When an API call fails, the client either receives a response or returns an error after its deadline or another failure condition. A timeout means the client stopped waiting; it does not prove that the server never received or acted on the request. That distinction matters most before retrying a payment, order, or other operation that changes state.
What happens when an API call fails?
The client sends a request and then waits for a response. The service may return success, return an error, or fail to respond before the client’s deadline. The client applies its own error-handling policy: it might return the error immediately, retry an eligible request, or stop because an attempt or time budget has been reached.
As an Amazon Associate I earn from qualifying purchases.
A timeout is an ambiguous outcome. The service may be slow, the network may be delayed, or the dependency may be unavailable. The request may also have reached the service and completed even though the response never reached the caller. Treat a timed-out state-changing request as potentially completed until you can determine otherwise.
How should you set timeouts?
Set a timeout for every outbound call before adding retries. Choose it as part of the full operation’s latency budget: the time the caller can wait, the service’s normal behavior, and the cost of holding connections, threads, or other resources. A timeout that is too short can reject work that would have completed; one that is too long can tie up resources during an outage. There is no universal timeout value for all APIs.
Where the client or platform supports them, distinguish connection establishment, TLS negotiation, response/read, per-attempt, and overall operation deadlines. Retry delays and attempt counts must fit inside the end-to-end deadline, not extend it without limit. For a product-specific example, AWS Well-Architected guidance dated 2023-04-10 gives an AWS API Gateway integration timeout range of 50 milliseconds to 29 seconds and says clients must implement their own retries for those integration timeouts. Those figures describe that AWS service guidance, not a general API prescription. AWS REL05-BP05.
When should you retry a failed API call?
Retry when the failure may be temporary and another attempt could succeed. Temporary network problems, service unavailability, and throttling can be candidates, depending on the API contract. An invalid request or authorization failure usually needs a correction rather than another identical request. Error classification varies by service and client: AWS SDKs, for example, classify failures as transient, throttling, or non-retryable, but not every API client retries automatically. AWS SDK retry behavior; Microsoft transient fault handling.
For writes and other state changes, establish that repeating the operation is safe before retrying. Use an idempotent operation or an API-supported idempotency mechanism where available. Otherwise, a second request can create duplicate effects or partial updates. AWS’s retry-with-backoff guidance likewise calls for idempotent operations when retries are used. AWS retry with backoff pattern.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Bound attempts and honor server timing
Set a finite attempt limit and a total retry budget. A per-request cap alone may not be enough: many callers retrying at once can collectively increase load on a struggling dependency. Check whether an SDK, proxy, gateway, and application are all retrying, since layered policies can multiply attempts.
Rank #3
If the API returns retry timing, follow that API’s documented semantics. Microsoft’s guidance says to honor a Retry-After response for at least the specified duration. AWS SDK documentation describes separate AWS-specific behavior for x-amz-retry-after; the two header contracts should not be treated as interchangeable. Microsoft transient fault handling; AWS SDK retry behavior.
Use backoff and jitter to pace retries
Exponential backoff increases the wait between attempts. Jitter randomizes that wait so clients are less likely to retry in synchronized bursts. Avoid repeated immediate retries: Microsoft’s guidance says not to perform more than one immediate retry and generally recommends exponential backoff with jitter for background work. Interactive requests may need a different policy to meet their latency budget.
AWS SDK documentation gives examples that are specific to its documented retry behavior: a 50 ms transient-error base delay, a 1,000 ms throttling base delay, a maximum computed backoff delay of 20 seconds, and a worked example with three maximum attempts. These are AWS SDK examples, not universal tuning values. The AWS Step Functions retry example in AWS Prescriptive Guidance uses a maximum of three retries and a 1.5-second increase multiplier; that, too, applies to that example rather than to all APIs. AWS SDK retry behavior; AWS retry with backoff pattern.
Free tools Windows power users keep installed
One-click scans. No signup required.
When do you need a circuit breaker?
Retries address a failure that might clear on another attempt. A circuit breaker addresses a dependency that is failing often enough that continuing to call it wastes resources or adds pressure. As Microsoft puts it, “The Circuit Breaker pattern serves a different purpose than the Retry pattern.” Microsoft Circuit Breaker pattern.
Best Value
Closed, open, and half-open states
- Closed: Calls flow normally while the breaker tracks outcomes.
- Open: After a configured failure threshold, the breaker rejects or fails calls quickly rather than sending them to the unhealthy dependency.
- Half-open: After an interval, the breaker permits a limited number of probes. Success closes it; failure opens it again. Limiting probes avoids flooding a service that is only beginning to recover.
Choose the threshold, open interval, and probe limit for the protected operation and its recovery behavior. A breaker is not proof that the remote service has recovered: an interval that is too long can keep calls blocked after recovery, while one that is too short can send repeated probes to a service that remains unhealthy. Retry logic should stop when the breaker indicates the failure is not transient.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How do you choose a failure policy?
Before setting numbers, consider the operation’s failure mode, safety, workload, and downstream condition. A background job that can be queued may tolerate a different wait strategy from an interactive request with a strict user-facing deadline.
- Failure type: Is it plausibly transient, throttling, a permanent request or configuration error, or an ambiguous timeout?
- Repeat safety: Can the operation be repeated without duplicate or partial effects?
- Latency budget: What are the per-attempt timeout, retry delays, total elapsed time, and caller deadline?
- Dependency health: Could more traffic worsen capacity or delay recovery?
- Workload: Is this an interactive request, background work, or work that can be queued or dead-lettered?
- Recovery controls: What are the attempt cap, aggregate retry budget, breaker threshold and open interval, and half-open probe limit?
- Retry ownership: Which of the SDK, proxy, gateway, or application owns retries?
Microsoft notes that retry policy depends on the resource and operating conditions; AWS guidance also emphasizes timeouts and limits on calls. Neither supports one numeric policy for every application. Microsoft transient fault handling; AWS REL05-BP03.
How should you implement and monitor the policy?
- Configure an explicit timeout for each outbound call and an overall deadline for the operation.
- Classify errors using the API’s documented behavior; do not retry permanent request or configuration failures unchanged.
- Verify that repeating a write is safe before enabling retries for it.
- Set finite attempt and total retry budgets. Use backoff and jitter where appropriate, and follow server-directed timing as the API specifies.
- Add a circuit breaker if repeated calls to a persistently failing dependency would waste resources or worsen the incident.
- Monitor timeouts, failures by category, retry counts, and end-to-end elapsed time. Review rising or regular retry counts as possible signs of degraded performance or availability.
Microsoft recommends monitoring failures, average retries, and elapsed time; AWS Well-Architected guidance calls out remote timeouts, persistent errors, call error rates, service-level objectives, and latency outliers. Microsoft transient fault handling; AWS REL05-BP05.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
