Use bounded retries for transient errors, a timeout for each request attempt, and a deadline for the whole operation. For any call that changes external state, reuse one idempotency key across retries when the API supports it. A timeout only means your client stopped waiting—it does not prove the server failed to act. If the result is uncertain and the provider offers no replay protection, reconcile the action before trying again.
Why a timeout can still lead to a duplicate action
A client timeout tells you that the response did not arrive within the time you allowed. It does not tell you whether the remote service received the request or completed it. A payment, message, or record creation may have succeeded even if the response was lost.
As an Amazon Associate I earn from qualifying purchases.
That makes retrying a timed-out mutation different from retrying a read-only request. Before replaying an action, determine whether the API guarantees that repeated requests are safe. Google Cloud warns that repeatedly executing non-idempotent operations can create side effects such as duplicate resources: Google Cloud’s retry strategy.
Recommended Free Tools
Classify the operation before choosing a retry policy
- Read-only: A repeated request should not change state, though it may still consume quota or incur cost.
- Naturally idempotent: Repeating the same operation has the same intended effect as performing it once. Confirm that behavior in the target API’s documentation.
- Mutating and protected: The provider supports an idempotency mechanism, allowing retries to refer to the same logical action.
- Mutating and unprotected: A timeout can leave the outcome unknown. Reconcile against the target system before any replay.
Do not let an agent’s language model decide on its own to repeat an uncertain side effect. Give the application control of the operation ID, retry policy, and reconciliation step.
#1 Best Overall
Set both an attempt timeout and an overall deadline
A per-attempt timeout limits how long one network call may wait. It does not limit the complete operation if the application makes several attempts and pauses between them. Set an overall deadline that includes request time, backoff waits, and retries the installed SDK may perform internally.
Before adding an application retry loop, inspect the SDK’s behavior for the installed version and endpoint. Nested SDK and application retries can multiply attempts, extending latency and increasing load. OpenAI’s Agents SDK model documentation is one place to check SDK-specific behavior; defaults can vary by client, version, and configuration.
Retry transient failures selectively
Retry only failures classified as transient for the specific API. A rate limit or temporary service problem may be recoverable; a quota, billing, authentication, or invalid-request error generally requires corrective action rather than another identical attempt. OpenAI’s guidance for API rate limits and 429 errors explains that rate-limit responses and exhausted quota require different responses.
For OpenAI API requests, follow a valid Retry-After value when the header is present. If there is no valid server delay, use exponential backoff with jitter: increase the wait between attempts while randomizing it to avoid many clients retrying in sync. OpenAI also advises stopping and deferring the request if a valid server delay exceeds the configured maximum retry delay; retrying sooner would disregard the server’s instruction. See OpenAI’s rate-limit guidance.
Bound retries by both attempt count and total elapsed time. If the next wait or request would exceed the deadline, stop or defer instead of allowing the operation to run indefinitely.
Use one idempotency key for one logical mutation
When the API supports idempotency keys, create a stable key for the logical action and reuse it on every attempt. Do not mint a new key merely because a request timed out: a new key can make the retry look like a separate action. Keep the mutation parameters identical across attempts and follow the provider’s documented key scope and retention period.
Stripe documents idempotency as a way to retry requests without accidentally performing the same operation twice. Its keys may be pruned after they are at least 24 hours old, so that retention detail is specific to Stripe and should not be assumed for other APIs. See Stripe’s idempotent requests documentation.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minutePC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Reconcile when replay protection is unavailable
For an unprotected mutation with an uncertain outcome, record a stable operation ID before sending the request. If the response is lost or times out, query the target system using a stable external reference or another reconciliation method supported by that system. Replay only after establishing that the original action did not complete. If the outcome remains unknown, stop automatic replay and surface the case for review.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Implement the control flow
- Create an operation ID. Assign it before the tool call and preserve it across worker restarts and retries.
- Choose replay handling. Identify whether the call is read-only, naturally idempotent, or mutating; for mutations, verify the provider’s idempotency support and scope.
- Set budgets. Configure a timeout per attempt and a deadline for the complete operation, including SDK retries and backoff delays.
- Classify failures. Retry only known transient errors. Respect a valid
Retry-After; otherwise use capped exponential backoff with jitter. - Preserve the action identity. For a protected mutation, reuse the same idempotency key and parameters on every attempt.
- Stop safely. If there is no server-side idempotency protection and the outcome is ambiguous, reconcile before replaying. Defer or request human review if the result cannot be established.
- Log the decision trail. Record the operation ID, attempt number, timeout, error class, chosen delay, and final outcome. Keep secrets and sensitive request bodies out of logs.
Illustrative pseudocode
operation_id = stable_id_for_this_logical_action
idempotency_key = stable_key(operation_id)
deadline = now() + total_budget
for attempt in 1..max_attempts:
if now() >= deadline:
stop_or_defer(operation_id)
result = call_api(
timeout = min(per_attempt_timeout, deadline - now()),
idempotency_key = idempotency_key,
same_mutation_parameters = true
)
if result.success:
record_success(operation_id, result)
return result
if not is_retryable_transient_failure(result):
surface_failure(operation_id, result)
return
delay = valid_retry_after(result) or exponential_backoff_with_jitter(attempt)
if now() + delay >= deadline:
stop_or_defer(operation_id)
sleep(delay)
if outcome_is_ambiguous(operation_id):
reconcile_before_any_replay(operation_id)This is illustrative control flow, not tested code. Providers and SDKs differ in retry defaults, error classifications, timeout behavior, idempotency support, and key retention; check the documentation for the exact client version and endpoint you use.
Quick Recap
What to verify in an SDK or provider
- Which transport failures and status codes are retried, and whether retries are enabled by default.
- Whether a valid server retry hint is honored and what happens when its delay exceeds the configured limit.
- How backoff, jitter, maximum attempts, and total request deadlines are configured.
- Whether streams or side-effectful operations can be replayed safely.
- How idempotency keys are scoped, whether request parameters must match, and how long keys are retained.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
