Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Some links on this page are affiliate links: if you buy through them we may earn a commission, at no extra cost to you.

Use a requests.Session with an HTTPAdapter configured with urllib3’s Retry policy. Set finite retry limits, a connect/read timeout, and an allowlist of methods that are safe to repeat. This handles transient connection failures and selected HTTP responses without an unbounded retry loop.

Configure retries for a Requests session

Requests does not retry failed connections by default, as its API documentation explains. Mount an adapter on both HTTP schemes so the policy applies to either kind of URL:

import requests
from requests.adapters import HTTPAdapter
from urllib3.util import Retry

retry = Retry(
    total=4,
    connect=4,
    read=2,
    status=3,
    backoff_factor=0.5,
    backoff_jitter=0.2,
    status_forcelist=(429, 500, 502, 503, 504),
    allowed_methods=frozenset({"GET", "HEAD", "OPTIONS"}),
    respect_retry_after_header=True,
)

session = requests.Session()
adapter = HTTPAdapter(max_retries=retry)
session.mount("http://", adapter)
session.mount("https://", adapter)

response = session.get("https://api.example.com/data", timeout=(3.05, 15))
response.raise_for_status()
print(response.json())

The values are an example policy, not a universal fit. Replace the example URL with the endpoint you call, and tune the limits and status codes to that API’s contract. total places an overall ceiling on retries; the connect, read, and status values set category-specific ceilings. Keeping both an overall and category limit makes the behavior finite and easier to reason about.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

What the adapter does

HTTPAdapter is Requests’ transport adapter; passing it a Retry object lets urllib3 apply retry rules to requests made through the session. A session reuses this configured behavior across calls. Mounting separately for http:// and https:// avoids accidentally applying it to only one scheme.

Why the timeout is still necessary

Retries and timeouts address different problems. The tuple (3.05, 15) gives a connect timeout and a read timeout in seconds. A timeout bounds how long a particular connection or read can wait; retries determine whether another attempt follows an eligible failure. urllib3 notes that read timeout measures the interval between socket reads, not necessarily the total time to receive a complete streamed response. For long-running or streamed responses, a read timeout alone is therefore not a complete end-to-end deadline. See the Requests timeout guidance.

Choose which requests and responses are retryable

Retry only operations that are safe to repeat

The example permits GET, HEAD, and OPTIONS. urllib3’s default set is idempotent methods: GET, HEAD, PUT, DELETE, OPTIONS, and TRACE. An operation is idempotent when repeating it has the same intended effect as performing it once. That property matters because a client may lose the response after a server has already performed the operation.

Do not casually add POST to allowed_methods. If a POST reaches the server but its response is lost, a retry could create a duplicate payment, job, or record. Retry POST only when the API provides an idempotency mechanism and your code uses it correctly. Method safety is an API-level decision, not something a retry library can infer. urllib3 documents method matching in its Retry reference.

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Select transient status codes intentionally

status_forcelist asks urllib3 to retry a response only when both conditions hold: the method is in allowed_methods, and the status code is in the list. The example includes 429 (rate limiting) and common server-error responses 500, 502, 503, and 504. Do not assume every API treats every one of these as transient; follow the service’s documented behavior. Client errors such as authentication or validation failures generally need a corrected request, not another identical attempt.

Honor Retry-After

With respect_retry_after_header=True, urllib3 uses a server-provided Retry-After delay when present for applicable responses, before falling back to its backoff behavior. This is useful for rate limits and temporary service unavailability: the server can indicate when it expects a retry rather than having every client choose its own immediate retry time. Keep retry budgets finite even when honoring this header.

Set backoff without creating a retry storm

Immediate retries can make an overloaded service worse, especially when many clients fail together. Exponential backoff spaces attempts out. urllib3 calculates the base delay from backoff_factor * 2**previous_retries; with a factor of 0.5, successive retry delays grow exponentially. The optional backoff_jitter=0.2 adds uniform jitter, varying the delay so clients are less likely to retry in lockstep. The backoff_max setting caps the backoff; choose a cap that fits the operation’s latency budget and API guidance. urllib3’s default backoff factor is zero, so set it deliberately if delayed retries are wanted. Details are in the urllib3 Retry reference.

Backoff does not itself define the maximum time your user or job can wait. Consider the combined effects of timeouts, retry count, server-directed waits, and backoff when setting an overall deadline. A request that is safe to repeat may still be inappropriate to retry synchronously if the caller cannot tolerate the wait.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Apply the policy to a website screenshot API

The same Requests pattern works when the endpoint is an HTTP API. For example, a developer can use it for a screenshot request to ScreenshotNeo. The retry policy handles eligible transport failures or listed HTTP statuses; it does not turn a successful HTTP response into a successful page capture. Check the response and the API’s documented result indicators for the outcome of the capture.

import requests
from requests.adapters import HTTPAdapter
from urllib3.util import Retry

retry = Retry(
    total=4,
    connect=4,
    read=2,
    status=3,
    backoff_factor=0.5,
    backoff_jitter=0.2,
    status_forcelist=(429, 500, 502, 503, 504),
    allowed_methods=frozenset({"GET"}),
    respect_retry_after_header=True,
)

session = requests.Session()
adapter = HTTPAdapter(max_retries=retry)
session.mount("http://", adapter)
session.mount("https://", adapter)

response = session.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
    timeout=(3.05, 15),
)
response.raise_for_status()
with open("shot.webp", "wb") as image_file:
    image_file.write(response.content)

Use the API key belonging to your account and avoid printing it in logs. ScreenshotNeo supports a one-call GET request and returns a PNG, JPEG or WebP screenshot, or a PDF, depending on the request. The ScreenshotNeo documentation covers request options and response behavior.

Or skip the browser setup

For a screenshot workflow, a direct API request avoids setting up and maintaining a browser capture stack. ScreenshotNeo accepts and removes cookie-consent banners before capture and removes 60+ known consent platforms, newsletter popups, and chat widgets; each of these steps can be turned off. Bot checks or CAPTCHAs, blank pages, timeouts, failed loads, and cache hits are not billed, and responses indicate the page verdict and billing status in headers. Its MCP server provides take_screenshot, get_page_info, and capture_pdf tools for AI agents using Claude, Cursor, or another MCP client. The free plan includes 1,000 shots per month with no card; paid plans start at $5 for 3,000.

import requests

r = requests.get(
    "https://api.screenshotneo.com/v1/shot",
    params={"access_key": "YOUR_API_KEY", "url": "https://stripe.com"},
    timeout=90,
)
open("shot.webp", "wb").write(r.content)

See the API documentation for parameters, output formats, and response details. Sign up for 1,000 free screenshots a month with no card.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Use other retry approaches when the scope differs

urllib3 directly

If your application already uses urllib3 rather than Requests, its retry policy can be configured at the pool or request level. That is a natural fit when you want urllib3’s HTTP-aware method, status, redirect, and Retry-After controls without adding a Requests session. Consult the urllib3 Retry reference for the API in your installed version.

Tenacity for broader operations

Tenacity can retry broader Python operations using decorator-based policies, including fixed, exponential, and randomized waits. It may suit a unit of work that includes HTTP plus parsing, queue access, or other I/O. It does not replace HTTP-specific decisions: you still need to know whether the method is safe to repeat, which response codes are transient, and how to honor server retry guidance.

Troubleshoot retries that do not behave as expected

  • The request fails once with a connection error. Confirm the call uses the configured session, not module-level requests.get(), and that the adapter is mounted for the URL’s scheme. Requests has no retry behavior by default.
  • A 429 or 503 response is returned without another attempt. Check that the status is in status_forcelist and that the method is allowed. Also verify the response status and API contract; retry rules do not apply indiscriminately to every response.
  • A POST may be duplicated. Remove POST from the allowlist unless the operation has an explicit idempotency design. If the server could have completed the first attempt, a lost response does not prove the operation failed.
  • The request still takes too long. Set connect and read timeouts on the call. Account for the time spent across multiple attempts and any server-directed or backoff sleeps; the read timeout is not necessarily a total-response deadline.
  • Retries occur immediately. Set a nonzero backoff_factor. Add jitter where supported by the installed urllib3 version, and consider a cap with backoff_max.
  • Retry configuration raises an argument error. Check the Requests and urllib3 versions installed in the environment and the parameter names supported by that urllib3 version. In particular, retry APIs have evolved; use the current allowed_methods spelling documented by urllib3.
  • The screenshot API returns a response, but the page is blank or blocked. An HTTP retry policy addresses transport errors and selected status codes, not the quality or contents of a capture. Inspect the page-verdict and billing headers and adjust capture options or the target URL as appropriate.

Log the final outcome safely

When a retry budget is exhausted, record the operation identifier or sanitized URL, final exception or response status, and attempt context needed to diagnose the issue. Do not log access keys, authorization headers, cookies, or other secrets. Avoid adding a second application-level retry loop around an adapter without accounting for the multiplied attempts and total wait.

Frequently Asked Questions

Does Requests retry failed requests by default?

No. A retry policy must be configured, for example with urllib3’s Retry object on a Requests HTTPAdapter.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Can I retry POST requests?

Only when repeating the operation is safe under the API’s idempotency design; otherwise a retry can duplicate a side effect.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.