Retry jitter spreads requests over time when many microservice clients fail together. Use it with capped exponential backoff to reduce synchronized retry bursts—but retry only plausibly transient failures, set firm limits, and protect operations against duplicate effects.
Why retries create a thundering herd
When a service becomes slow or unavailable, many clients may fail at nearly the same time. If each client retries after the same fixed delay, they can send another wave of requests together. That burst adds load just as the service is trying to recover.
As an Amazon Associate I earn from qualifying purchases.
Exponential backoff increases the wait between successive attempts, which slows repeated requests. But when clients share the same schedule, backoff alone can preserve clusters: clients that failed together may keep waking and retrying together. Jitter adds a random element to each wait, spreading those attempts across time. Backoff slows retries; jitter reduces their alignment.
As Marc Brooker put it in the AWS Architecture Blog, “The solution isn’t to remove backoff. It’s to add jitter.” The 2015 article’s simulation compared 100 contending clients and found that full jitter reduced call count by more than half versus unjittered exponential backoff. That is a result from that simulation, not a guarantee for every production workload.
#1 Best Overall
Choose a jitter strategy for your workload
Jitter strategies make different trade-offs between predictable delay and how widely retries spread. Brooker’s comparison covers three common approaches:
| Strategy | How it varies the wait | Trade-off in the AWS comparison |
|---|---|---|
| Full jitter | Chooses a random wait from zero up to the current capped backoff interval. | Similar client work to equal jitter; reduced calls by more than half versus unjittered backoff in the 100-client simulation. |
| Equal jitter | Retains part of the backoff interval while randomizing the remainder, so waits do not fall to zero. | Similar client work to full jitter in the article’s comparison. |
| Decorrelated jitter | Varies the next delay in relation to the previous delay rather than selecting from a fixed exponential interval. | Higher client work than full and equal jitter in the article’s comparison. |
These results do not make one strategy universally best. Consider request volume, concurrency, acceptable latency, and how much spread the service needs. A policy that delays some clients longer may reduce a burst but use more of their latency budget.
Build a retry policy with safeguards
Retry only plausibly transient failures
Retries are appropriate for failures that may resolve on another attempt, such as throttling, timeouts, or transient service errors. Fail fast on persistent problems such as permission or configuration errors; repeating a request cannot fix them. When a service supplies guidance such as a Retry-After header, follow it where applicable.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minutePC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Bound attempts, time, and delay
Use exponential backoff with jitter, a maximum attempt count, an overall elapsed-time budget, and a delay cap. Longer waits can push a request beyond the caller’s latency budget, so decide how long the caller can wait before configuring retries. A delay cap limits the wait for any one retry; it does not replace a total-attempt limit or elapsed-time budget.
Prevent duplicate side effects
A timeout does not prove that the server failed to carry out the first request. It may have completed the operation while its response was lost. Retrying a non-idempotent action—such as creating a charge or submitting an order—can therefore repeat its effect. Make the operation idempotent or use an idempotency mechanism before retrying it.
Choose one deliberate retry layer
Retries can be configured in an SDK, a service client, an application, or several layers at once. If each layer retries independently, one user request can produce many downstream attempts. Inspect the actual SDK and library behavior rather than assuming a built-in retry policy is enabled or disabled.
Rank #4
Test and observe retry behavior
Exercise timeouts, throttling, transient failures, and persistent errors in resilience tests. Check that retries spread out, stop at the configured limits, and do not repeat unsafe effects. Monitor repeated failures and retry behavior: retries may help with rare transient errors, but they can worsen overload when a dependency is already under pressure.
Free tools Windows power users keep installed
One-click scans. No signup required.
A concrete example: AWS SDK standard mode
AWS documents standard retry mode as using exponential backoff with full jitter. Its reference formula is random(0, 1) × min(20,000 ms, base_delay × 2^retry). The documented base delay is 50 ms for transient non-throttling errors and 1,000 ms for throttling errors; the maximum individual delay is 20 seconds. The reference describes three default total attempts: one initial request plus two retries. These are AWS SDK reference values, not settings for microservices generally; verify the exact SDK and version used by your workload in the AWS SDKs and Tools Reference Guide.
Best Value
The same AWS reference describes a retry quota that can stop retries when its tokens are depleted. Its stated aim is to avoid spending time on retries unlikely to succeed and reduce retry traffic during disruption. When evaluating any SDK or library policy, check its retryable-error classification, jitter algorithm, delay cap, total-attempt semantics, retry budget or quota, and whether it treats throttling differently.
Quick Recap
Questions to answer before enabling retries
- Which errors are genuinely transient for this dependency, and which should fail immediately?
- What attempt limit, elapsed-time budget, and delay cap fit the caller’s latency requirements?
- Can the operation safely run more than once, or does it need an idempotency mechanism?
- Which layer owns retries, and could another layer multiply downstream attempts?
- How will tests and monitoring reveal retry bursts, repeated failures, or retries that worsen overload?
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
