A Python requests.Session keeps cookies and reuses connections across related requests. It does not keep your proxy exit IP the same. If a login, cart, or multi-step form must stay on one network identity, you need a sticky session from your proxy provider, configured alongside the Session. If you are collecting independent pages or records, rotation between units of work is usually the better fit. This guide separates the two layers, shows how to pass proxy credentials safely, and explains how to choose between sticky and rotating behavior.
Two layers: client state and proxy identity
Most confusion about session persistence comes from treating two different features as one. The Requests library and your proxy provider each control something different.
The Requests Session: client state
The Requests Advanced Usage documentation says that a Session “allows you to persist certain parameters across requests.” It also states that the Session “persists cookies across all requests made from the Session instance, and will use urllib3’s connection pooling.” Those are the guarantees you get from requests.Session(): cookies, default headers and proxy settings carried between calls, and pooled TCP connections.
As an Amazon Associate I earn from qualifying purchases.
What a Session does not guarantee is that the proxy will send each request out through the same IP address. Whether the exit IP stays fixed is decided by the proxy service, not by Requests.
Provider sticky sessions: network identity
A sticky session is a feature of the proxy provider. Some providers keep a client on the same exit IP for a period of time when you present a session identifier or a provider-specific credential format. Others do not offer this at all. The duration, the identifier syntax, and whether the feature is available on your plan all vary by provider, so confirm them in that provider’s current documentation before you build a workflow around them.
Provider rotation: a new exit per unit of work
Rotation means the provider assigns a different exit IP over time or per request. Requests does not implement a rotation schedule. If you need rotation, you decide the boundary yourself, for example one exit per product page, per account, or per batch of independent records, and then use a provider setting or a new connection to get the new exit.
#1 Best Overall
- 【WIRELESS MOBILE MINI TRAVEL ROUTER】 Convert a public network (wired or wireless) to a private Wi-Fi for secure surfing. Tethering. Powered by any laptop USB, power banks or 5V/2A DC adapters (sold separately). 39g (1.41 Oz) only, portable and pocket friendly. 2.4GHz ONLY
- 【OPEN SOURCE & PROGRAMMABLE】 OpenWrt pre-installed, USB disk extendable.
- 【LARGER STORAGE & EXTENDABILITY】 128MB RAM, 16MB Flash ROM, dual Ethernet ports, UART and GPIOs available for hardware DIY.
- 【OPENVPN CLIENT】 OpenVPN client pre-installed, compatible with 30+ VPN service providers.
- 【PACKAGE CONTENTS】 GL-MT300N-V2 (Mango) mini router (2-year Warranty), USB cable, Ethernet cable, User Manual. Please update to the latest firmware.
Configuring proxy authentication in Requests
- Get the endpoint and credential format from your provider. Copy the host, port, and authentication format exactly as the provider documents them. Do not assume a username pattern from another service.
- Keep the secret out of your code and repository. Load the proxy URL at runtime from a secret store or a deployment secret, not from a hard-coded string or a committed
.envfile. - Build the proxy dictionary explicitly. Pass
proxies=to the Session or to each call so the routing is visible in your code. - Set timeouts. Use
timeout=(connect, read)so a stalled proxy does not hang the job. - Verify the exit IP before relying on it. Send a request through the same Session to an IP-echo endpoint you control, then repeat it later in the same flow to check whether the identity held.
import os
import requests
proxy_url = os.environ["PROXY_URL"] # Populated at runtime from a secret store; never hard-coded.
proxies = {"http": proxy_url, "https": proxy_url}
with requests.Session() as session:
response = session.get(
"https://example.com/",
proxies=proxies,
timeout=(5, 30),
)
response.raise_for_status()
This example shows a persistent client session routed through a proxy. It does not create a sticky session. The proxy URL you put in PROXY_URL must follow your provider’s documented format. If the provider puts a session identifier or a country selection into the username, that syntax comes from the provider, not from Requests.
Credentials in the proxy URL
The Requests documentation shows Basic proxy authentication in the form http://user:pass@host:port/. If your username or password contains characters such as @, :, or /, percent-encode them first, for example with urllib.parse.quote, or the URL will be parsed incorrectly.
HTTPProxyAuth
Requests also provides requests.auth.HTTPProxyAuth, which the Developer Interface documentation describes as a way to attach “HTTP Proxy Authentication to a given Request object.” It handles the proxy hop only. It is not authentication to the destination website, and you should not use it to log in to the site you are scraping. Test it against your provider before depending on it for HTTPS traffic, because the proxy-URL form is the one the Requests advanced documentation covers for proxy credentials.
Rank #2
- 【Advanced Home Data & Media Hub】For advanced home users who need phone backup, file storage, and centralized data management. Centralize family photos, 4K videos, movies, computer backups, and personal files in one place while running multiple apps for home entertainment and everyday data management. Suitable for households with growing digital libraries and multiple NAS use cases.
- 【Built for Creators, Media Servers & Advanced Apps】Powered by the Intel N100 Quad-Core CPU, 8GB DDR5 RAM, 2.5GbE networking, and dual M.2 NVMe slots, DXP2800 handles large files and heavier workloads with ease. Run Docker, virtual machines, and media server applications compatible with Plex—ideal for content creators, tech enthusiasts, and advanced home users managing 4K videos, RAW photos, personal media libraries, and multiple NAS apps.
- 【Up to 80TB for Growing Digital Libraries】 Supports up to 80TB of storage using two HDD bays and two M.2 NVMe SSD slots for family photos, movies, RAW photos, 4K videos, work files, and device backups. AI photo management supports recognition of people, objects, scenes, and locations, album organization, and duplicate photo detection. HDDs and SSDs are not included.
- 【AI-powered Home Surveillance】Turn DXP2800 into a centralized home surveillance hub by connecting compatible network cameras and storing recordings locally on your NAS. AI-powered features include Face Recognition, People Detection, and Pet Detection, helping advanced home users review important events more efficiently while managing home surveillance and personal data in one place.
- 【One data Center Across Your Devices】Keep files from desktops, laptops, phones, tablets, and other devices together instead of scattered across cloud accounts and external drives. Access, back up, organize, and share data across Windows, macOS, Android, iOS, web browsers, and compatible smart TVs—ideal for creators and advanced home users working across multiple devices.
Sticky or rotating: how to choose
| Workflow | Better starting point | Why | Caveat |
|---|---|---|---|
| Login, cart, checkout, or a multi-step form where each step depends on the last | Provider sticky session plus one requests.Session |
The flow relies on cookies and on the server seeing one network identity | The Session alone does not pin the exit IP. Confirm the provider’s stickiness duration and identifier syntax. |
| Independent pages or records that can each be fetched on their own | Provider rotation between independent units | A changed exit IP between unrelated tasks does not break a dependent sequence | Set the rotation boundary yourself and respect the target site’s rules. Provider rotation policies differ. |
| Debugging unexpected routing | Explicit proxies= plus a check of the environment |
Request-level configuration removes ambiguity from inherited settings | Environment proxy variables can still affect behavior unless you turn them off. |
The comparison axes that matter most are whether the work is dependent or independent, how long the provider keeps an exit IP, where the rotation boundary falls, which geographic choices are available, and the provider’s documented authentication format. Requests can report none of these values for you; they come from the provider.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Environment variables and proxy precedence
Requests can read the standard proxy environment variables, including http_proxy, https_proxy, no_proxy, and all_proxy, along with their uppercase forms. Those settings apply when the proxy is not set on the request. This is convenient in most deployments, but it makes routing harder to reason about when several layers set proxies.
Rank #3
- One Place for All Your Data - Consolidate scattered files from multiple computers, phones and external drives into one accessible hub with 100% ownership
- Professional File Collaboration - Share projects with clients, sync documents across teams and maintain version control without Dropbox fees
- Automated Backup Protection - Set-and-forget backups for Macs, PCs and mobile devices to multiple destinations including cloud and external drives
- DIY Surveillance System - Transform IP cameras into a professional monitoring solution with motion alerts, recording schedules and remote viewing
- 2-Year Warranty - Reliable hardware backed by Synology's expert customer support team and ongoing software updates
To keep behavior deterministic, pass proxies= on each call or set session.proxies explicitly, and check the environment when a request goes somewhere unexpected. If you need to ignore inherited settings entirely, Requests exposes session.trust_env; setting it to False stops the Session from reading environment proxy and netrc configuration.
Python’s urllib.request documentation also notes that HTTP_PROXY is ignored when REQUEST_METHOD is set. That behavior exists to protect CGI environments, where a client-supplied header can become an environment variable. If your code runs under CGI or a similar wrapper, expect the uppercase variable to be ignored there.
Quick Recap
Best Value
- Complete Phone & Computer Backup - Automatically protect photos, documents and videos from iPhone android, Mac and Windows to one secure location
- Your Private File Cloud - Access files from anywhere and share large projects with family or clients without relying on expensive cloud subscriptions
- Smart Home Security Hub - Monitor your home 24/7 with AI-powered surveillance that detects people, vehicles and sends instant alerts
- 100% Data Ownership - Keep full control of your personal data with multi-platform access and no monthly subscription fees
- 2-Year Warranty - Reliable hardware backed by Synology's expert customer support team and ongoing software updates
Rank #4
- Unlimited bandwidth, unlimited data.
- Super-fast VPN and one tap connect.
- Free worldwide multiple servers.
- Works with all type of data carries. (Wi-Fi, 4G, LTE, 3G).
- No registration, sign up needed.
Credential and TLS safety
- Base64 is not encryption. The urllib3 utilities reference describes proxy Basic credentials as Base64-encoded bytes using a configured encoding. That is how Basic authentication is transmitted, not a protection. Anyone who can read the header can decode it, so use HTTPS to the proxy where the provider offers it.
- Do not log the proxy URL. Logging a full proxy URL writes the password to your logs. Log the host and the outcome instead.
- Do not set
verify=Falseto fix a proxy error. The Requests API documentation warns that this accepts untrusted, mismatched, or expired certificates and can expose the client to man-in-the-middle attacks. Fix the trust chain or the proxy configuration instead.
Troubleshooting common failures
- HTTP 407 Proxy Authentication Required. The proxy did not accept the credentials. Check the username format against your provider’s current documentation, confirm the password is percent-encoded, and confirm the credentials are attached to the proxy hop, not the destination site.
- Cookies are kept but the login still breaks mid-flow. The cookies are working, but the exit IP changed. Request sticky behavior from the provider for that flow, then repeat the exit-IP check from the procedure above.
- Requests goes out through an unexpected proxy. An environment variable or a Session-level setting is overriding your expectation. Pass
proxies=explicitly on the call and setsession.trust_env = Falseto confirm whether inherited settings were the cause. - The job hangs. Add a read and connect timeout. Without one, a stalled proxy can block the thread indefinitely.
Practical decision summary
- Use one
requests.Sessionfor any sequence that depends on cookies, whatever proxy you choose. - Use a provider sticky session when that sequence also depends on the same exit IP, and confirm the duration in the provider’s documentation.
- Use provider rotation for independent work, and choose the boundary so the target site’s rules are respected.
- Configure proxies explicitly, load credentials at runtime from a secret store, and verify the exit IP rather than assuming it.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
