How to Rotate IP Addresses Automatically in Python (2026)

Hard-coding one proxy into a scraper works for ten requests. It fails at request ten thousand. Target sites track request rates per IP, and a single address doing all the work is the easiest pattern in the world to spot: same IP, same headers, same cadence, thousands of pages. The result is familiar to anyone who has scraped at scale — a rising wall of 429s, then 403s, then a dead IP.

Automatic IP rotation solves this by spreading requests across a pool of addresses so no single IP carries a suspicious share of the traffic. Done well, it looks like ordinary distributed traffic. Done badly — rotating too fast, reusing dead IPs, or rotating mid-session — it creates problems worse than the one it solves. This guide covers the three rotation strategies that actually work in Python, with complete code for each, plus pool health checking and request logging so you can debug what really happened.

The Three Rotation Strategies

There is no single "best" rotation strategy. The right one depends on what you are scraping.

| Strategy | How it works | Best for | Watch out for | |---|---|---|---| | Per-request | New IP for every request | Large-scale public data collection, price monitoring | Breaks login sessions and multi-step flows | | Timed | Same IP for a fixed window (e.g. 10 minutes), then rotate | Session-like flows, paginated browsing | A window that is too long concentrates load on one IP | | On-failure | Keep one IP until it gets blocked, then switch | Small jobs, sites with generous rate limits | A dying IP can burn many retries before you notice |

Most production scrapers combine all three: per-request rotation as the default, sticky windows for flows that need continuity, and on-failure switching as a safety net. The code below builds that layered setup piece by piece.

Strategy 1: Per-Request Rotation with itertools.cycle

The simplest robust pool is a cycle over your proxy list. Each request takes the next proxy in turn, and the pool wraps around forever. No threads, no state machines — just a queue that never empties.

import itertools
import requests

PROXIES = [
    "http://user:[email protected]:8000",
    "http://user:[email protected]:8001",
    "http://user:[email protected]:8002",
]

pool = itertools.cycle(PROXIES)

def fetch(url):
    proxy = next(pool)
    proxies = {"http": proxy, "https": proxy}
    resp = requests.get(url, proxies=proxies, timeout=15)
    resp.raise_for_status()
    return resp

for page in range(1, 6):
    r = fetch(f"https://example-shop.com/products?page={page}")
    print(page, r.status_code)

Two details matter here. First, always set a timeout: a dead proxy without one can hang a worker for minutes. Second, keep the pool as endpoint addresses, not as resolved IPs — with a rotating residential gateway, one endpoint can serve many exit IPs, and the gateway itself handles part of the rotation for you.

Strategy 2: Timed Rotation with Sticky Windows

Some tasks need the same IP for a while. Logging in, adding items to a cart, and checking out across three requests will fail if each request arrives from a different country. Timed rotation keeps one proxy for a window — commonly 1, 10, or 30 minutes — then moves on.

import time
import itertools
import requests

pool = itertools.cycle(PROXIES)
current_proxy = next(pool)
window_start = time.time()
WINDOW_SECONDS = 600  # 10-minute sticky window

def fetch_sticky(url):
    global current_proxy, window_start
    if time.time() - window_start >= WINDOW_SECONDS:
        current_proxy = next(pool)
        window_start = time.time()
    proxies = {"http": current_proxy, "https": current_proxy}
    resp = requests.get(url, proxies=proxies, timeout=15)
    resp.raise_for_status()
    return resp

Match the window to the task, not to a round number. A login-and-checkout flow needs minutes, not seconds; a product-listing crawl does not need stickiness at all. If your provider supports session control via username parameters (for example, appending a session id to the proxy username), prefer that over a client-side timer — the provider then pins the exit IP for you, even across process restarts.

Strategy 3: On-Failure Rotation with Backoff on 429 and 403

Rotation on a fixed schedule wastes good IPs and keeps bad ones too long. The smarter trigger is failure itself: when a proxy returns 429 (rate limited) or 403 (forbidden), retire it for that request and try the next one, with exponential backoff between attempts so you do not hammer a struggling site.

import time
import itertools
import requests

pool = itertools.cycle(PROXIES)
MAX_ATTEMPTS = 5

def fetch_with_retry(url):
    last_error = None
    for attempt in range(MAX_ATTEMPTS):
        proxy = next(pool)
        proxies = {"http": proxy, "https": proxy}
        try:
            resp = requests.get(url, proxies=proxies, timeout=15)
            if resp.status_code in (429, 403):
                # Back off: 1s, 2s, 4s, 8s ... and try the next proxy
                time.sleep(2 ** attempt)
                continue
            resp.raise_for_status()
            return resp
        except requests.RequestException as e:
            last_error = e
            time.sleep(2 ** attempt)
    raise RuntimeError(f"All {MAX_ATTEMPTS} attempts failed") from last_error

The backoff is as important as the rotation. Retrying instantly after a 429 teaches the target site that you ignore its limits, which is how a temporary rate limit becomes a permanent block. A 1-2-4-8 second ladder is polite, cheap, and usually enough. Also note the distinction the code makes: a 429 or 403 means "this IP is burned for now, move on," while a connection error may mean the proxy itself is down — both cases advance the pool, but only the first is a signal about the target site.

The Session-Per-Identity Pattern

The pattern that ties everything together is one session per identity. Instead of sharing a single requests.Session across your whole crawl, create a session bound to one proxy and one header set, and reuse it for that identity's entire flow. This keeps cookies, connection pooling, and IP consistent — exactly what a real browser session looks like.

import requests

class Identity:
    def __init__(self, proxy, user_agent):
        self.session = requests.Session()
        self.session.proxies = {"http": proxy, "https": proxy}
        self.session.headers.update({"User-Agent": user_agent})

    def get(self, url):
        resp = self.session.get(url, timeout=15)
        resp.raise_for_status()
        return resp

identities = [
    Identity(PROXIES[0], "Mozilla/5.0 (Windows NT 10.0; Win64; x64)"),
    Identity(PROXIES[1], "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)"),
]

# Each identity browses independently, with its own cookies and IP
for ident in identities:
    ident.get("https://example-shop.com/account")

One identity, one IP, one header set, one cookie jar. Mixing them — same cookies from two IPs, or one IP with ten user-agents — is a fingerprinting signal that rotation alone cannot fix.

Pool Health Checking: Don't Rotate Through Dead IPs

A rotation pool is only as good as its worst member. If a third of your proxies are dead, a third of your requests fail before retry logic saves them, and your crawl runs a third slower. Check the pool before the job starts and drop failures fast.

import concurrent.futures
import requests

CHECK_URL = "https://httpbin.org/ip"

def is_healthy(proxy):
    try:
        r = requests.get(CHECK_URL,
                         proxies={"http": proxy, "https": proxy},
                         timeout=10)
        return r.status_code == 200
    except requests.RequestException:
        return False

with concurrent.futures.ThreadPoolExecutor(max_workers=10) as ex:
    results = list(ex.map(is_healthy, PROXIES))

healthy = [p for p, ok in zip(PROXIES, results) if ok]
print(f"{len(healthy)}/{len(PROXIES)} proxies healthy")

Run this check on a schedule — at job start and every hour during long crawls — and log the results. A pool that degrades over a day tells you something about the provider; a pool that is dead at 9 a.m. tells you to stop the job before it burns its retry budget.

Logging Which IP Served Which Request

When a crawl fails, the first question is always "which IP was it?" Without logging, you cannot answer it. Wrap your fetch function so every request records its proxy, status, and latency in one line.

import logging
import time

logging.basicConfig(level=logging.INFO,
                    format="%(asctime)s %(message)s")

def fetch_logged(url):
    proxy = next(pool)
    start = time.time()
    try:
        resp = requests.get(url,
                            proxies={"http": proxy, "https": proxy},
                            timeout=15)
        logging.info(f"{resp.status_code} {url} via {proxy} "
                     f"in {time.time() - start:.2f}s")
        return resp
    except requests.RequestException as e:
        logging.warning(f"FAIL {url} via {proxy}: {e}")
        raise

This log is also your evidence when tuning: if one proxy shows a 40 percent failure rate while the rest sit at 2 percent, retire it. If failures cluster at a particular hour, your rate is too high for that window. Rotation without measurement is guessing.

Common Mistakes That Get Rotating Pools Blocked

First, rotating on every request for a logged-in flow. Sessions exist for a reason; breaking them mid-checkout looks automated because it is not how browsers behave. Second, rotating IPs but keeping one user-agent and one header order — the IP changes while the fingerprint stays identical, which is trivially linkable. Third, ignoring robots.txt and hammering a site at full speed just because the IPs rotate. Rotation spreads load; it does not license unlimited load. Respect crawl-delay, cache aggressively, and prefer an official API when one exists. Fourth, never verifying the exit IP. A misconfigured proxy variable silently sends traffic from your own server IP, and you discover it from a block, not from a test.

Putting It All Together

A solid default setup is: a health-checked pool, per-request rotation for public pages, sticky windows or session-per-identity for logged-in flows, on-failure switching with exponential backoff, and logging on every request. Start with that, measure failure rates per proxy and per hour, and tune only what the numbers say is broken. Rotation is infrastructure, not magic — it buys headroom, and polite request design spends it wisely.

FAQ

How often should I rotate my IP when scraping?

As slowly as the task allows. Per-request rotation fits large public crawls; logged-in or multi-step flows need a sticky window of minutes, and rotating mid-session breaks logins and carts. Rotate on failure (429/403) in all cases.

Does rotating IPs alone prevent blocks?

No. Sites also fingerprint headers, TLS behavior, request timing, and cookies. Rotate the IP together with a consistent per-identity header set, realistic delays, and backoff — and respect robots.txt and rate limits.

What is the difference between rotation and a sticky session?

Rotation changes the IP per request or per time window. A sticky session pins one exit IP for a chosen duration (for example 10 or 30 minutes) so a multi-step flow stays consistent. Many providers implement stickiness via a session id in the proxy username.

How do I know if my rotation is actually working?

Log the exit IP per request and check it against an echo endpoint such as httpbin.org/ip during testing. In production, track failure rates per proxy: a healthy pool shows evenly spread successes, not one IP doing all the work or one failing constantly.

How many proxies do I need for rotation?

It depends on volume and the target's tolerance, not on a fixed number. Start with a small pool, measure the failure rate, and grow it until failures flatten. A small healthy pool beats a large pool full of dead or flagged IPs.

Try It

Before wiring rotation into a real job, verify your setup end to end: check that each proxy in your pool actually returns a different exit IP, and that a sticky session holds one IP for its full window. Test your proxies at IPStation's free proxy test tool, and see residential and static ISP plans for rotation-friendly pools on the pricing page.