How to rotate proxies in Python, and why yours may not rotate

A rotating gateway changes the IP per connection, not per request. Most Python clients keep connections open, so they quietly reuse one IP. Here’s what rotates, measured.

7 min readTested on requests 2.34 · httpx 0.28 · Scrapy 2.19
Three requests through a rotating proxy: requests.get rotates, requests.Session, httpx.Client and Scrapy reuse one IP, and a session label per request rotates with any client.Does it rotate?3 requests each, one colour = one exit IPreq 1 req 2 req 3requests.get()new connection each timerequests.Session()same IP 3×httpx.Client()same IP 3×Scrapysame IP 3×+ session label per requestnew IP each time

In short

requests.get() rotates because it opens a new connection each time. requests.Session, httpx.Client and Scrapy keep connections open and send every request through the same exit. The fix that works with any client: a fresh session label in the proxy username for each request, sent to the sticky port.

requests.get · rotatesSession / httpx / Scrapy · same IP-session-<label> · rotates

A rotating residential gateway chooses an exit IP when a connection to it opens. For an HTTPS site, that connection becomes a tunnel: the client asks the proxy to CONNECT to the website once, then sends request after request through it. The gateway never sees those requests, so it has no chance to switch the IP.

That is fine when a client opens a new connection per request. But connection reuse is exactly what good HTTP clients are built for: it saves a TCP and TLS handshake every time. So the clients you’d pick for serious scraping are the ones that don’t rotate. We sent three requests through our gateway with each:

ClientExit IPsRotates?
requests.get()51.146.216.161 · 8.47.98.240 · 82.40.78.80Yes
requests.Session()209.94.95.37 · 209.94.95.37 · 209.94.95.37No
httpx.Client()76.135.82.93 · 76.135.82.93 · 76.135.82.93No
httpx.Client(keep-alive 0)142.173.227.6 · 38.97.214.33 · 100.37.193.236Yes
Scrapy37.133.72.35 · 37.133.72.35 · 37.133.72.35No
Fig. 1 — Three sequential requests per client to api.ipify.org through via.proxies.vision:9000, October 14, 2026.

A fresh session label per request (works everywhere)

On the sticky port (10000) the gateway pins an exit to a session label. Give each request its own label and it gets a new exit for each, whatever your client does with its connections. This is the one we recommend.

rotate.py
import os
import uuid

import requests

USER = os.environ["PROXY_USER"]
PASS = os.environ["PROXY_PASS"]
GATEWAY = "via.proxies.vision:10000"  # the sticky port: it reads a session label


def proxies(targeting: str = "") -> dict:
    """A fresh session label per call = a fresh exit IP, even on a kept connection."""
    user = f"{USER}{targeting}-session-{uuid.uuid4().hex[:8]}-lifetime-3"
    url = f"http://{user}:{PASS}@{GATEWAY}"
    return {"http": url, "https": url}


def get(session: requests.Session, url: str, attempts: int = 3, **kw) -> requests.Response:
    """Retry network errors: residential exits occasionally drop a connection."""
    targeting = kw.pop("targeting", "")
    for i in range(attempts):
        try:
            return session.get(url, proxies=proxies(targeting), timeout=30, **kw)
        except requests.exceptions.RequestException:
            if i == attempts - 1:
                raise


with requests.Session() as s:
    for _ in range(3):
        print(get(s, "https://api.ipify.org").text)  # a different IP each time

Or: a new connection per request

requests.get() without a session already does this. With httpx, turn keep-alive off: httpx.Client(proxy=url, limits=httpx.Limits(max_keepalive_connections=0)). Every request then pays for a new handshake, which is slower than the label approach.

Retry network errors

Residential exits are home connections and occasionally drop mid-request. In our runs that showed up as a ConnectTimeout, an SSLError or a RemoteProtocolError now and then. Retrying with a new label gets a healthy exit; the get() helper above does that.

Keeping one IP on purpose

For a login, a cart or anything that must look like one visitor: the same label on every request.

sticky.py
import os

import requests

USER = os.environ["PROXY_USER"]
PASS = os.environ["PROXY_PASS"]

# Same label = same IP for up to 10 minutes, across connections and processes.
user = f"{USER}-country-de-session-checkout1-lifetime-10"
proxy = f"http://{user}:{PASS}@via.proxies.vision:10000"

with requests.Session() as s:
    s.proxies = {"http": proxy, "https": proxy}
    for _ in range(3):
        print(s.get("https://api.ipify.org", timeout=30).text)  # the same German IP

lifetime-10 keeps the IP for up to ten minutes (up to 1440 is allowed). The label is what matters, not the connection: requests with the same label got the same German IP whether they shared a Session or not.

Many requests at once

A shared AsyncClient opens one connection per parallel request and then reuses them, so you get as many IPs as connections, not one per request.

fetch_async.py
import asyncio
import os
import uuid

import httpx

USER = os.environ["PROXY_USER"]
PASS = os.environ["PROXY_PASS"]


async def fetch(url: str) -> str:
    # One client per request with its own session label: a new exit for each.
    proxy = f"http://{USER}-session-{uuid.uuid4().hex[:8]}-lifetime-3:{PASS}@via.proxies.vision:10000"
    for attempt in range(3):
        try:
            async with httpx.AsyncClient(proxy=proxy, timeout=30) as client:
                return (await client.get(url)).text
        except httpx.TransportError:
            if attempt == 2:
                return "failed"
    return "failed"


async def main() -> None:
    sem = asyncio.Semaphore(10)  # at most 10 requests at a time

    async def limited(url: str) -> str:
        async with sem:
            return await fetch(url)

    ips = await asyncio.gather(*[limited("https://api.ipify.org") for _ in range(20)])
    print(f"{len(set(ips) - {'failed'})} distinct IPs for {len(ips)} requests")


asyncio.run(main())

In our run this printed 20 distinct IPs for 20 requests. The semaphore caps how many run at once; the label gives each its own exit.

Scrapy

Scrapy also keeps connections open. A downloader middleware that sets a new label per request fixes it.

middlewares.py
import os
import uuid


class RotatingSessionMiddleware:
    """A fresh session label per request, so every request leaves through a new IP.
    Scrapy keeps connections open; without the label it would reuse one exit."""

    def __init__(self):
        self.user = os.environ["PROXY_USER"]
        self.password = os.environ["PROXY_PASS"]

    def process_request(self, request, spider=None):
        if "proxy" not in request.meta:
            label = uuid.uuid4().hex[:8]
            request.meta["proxy"] = f"http://{self.user}-session-{label}-lifetime-3:{self.password}@via.proxies.vision:10000"
pythonsettings.py
DOWNLOADER_MIDDLEWARES = {
    "myproject.middlewares.RotatingSessionMiddleware": 350,
}

With the middleware, three requests from one spider left through three different IPs. Without it, all three used the same one.

Why does requests.Session use the same proxy IP for every request?
A Session keeps the connection to the proxy open and sends the next request through the same tunnel. A rotating gateway picks the exit when a connection opens, so every request after the first leaves through the same IP. Use requests.get without a session, or send each request with a fresh session label through the sticky port.
Does httpx rotate the proxy IP on every request?
Not with a shared client: httpx.Client and AsyncClient reuse pooled connections, so sequential requests share one exit IP and parallel requests get one IP per open connection. Disable keep-alive with httpx.Limits(max_keepalive_connections=0) or use a session label per request.
How do I keep the same IP for a login in Python?
Add a session label and a lifetime to the proxy username, for example USER-country-de-session-checkout1-lifetime-10, and connect to the sticky port. Every request with that label leaves through the same IP for up to ten minutes, across connections and even across processes.
Should I rotate through a list of proxies myself?
With a rotating gateway you don't need a list: one endpoint, and the gateway picks the exit. A list only helps with static proxies, and then the same connection-reuse rules apply.

Residential proxies, pay per GB

HTTP and SOCKS5. Country, city and sticky sessions in the username. No subscription.