How to Rotate Your IP Address in Python
There are two ways to rotate your IP in Python. The hard way is to maintain your own list of proxies and pick one at random per request, handling dead IPs and bans yourself. The easy way is to point one rotating gateway at your code so it assigns a new IP per request automatically. This guide shows both, plus retries and backoff, Scrapy, and async aiohttp, all with working, copy-paste code.
requests & aiohttp Scrapy middleware Retries & backoff
Why rotate your IP in Python at all?
If you are scraping, testing from different locations, or hitting an API with strict per-IP rate limits, sending everything from a single address gets that address throttled or blocked fast. Rotating the IP spreads your requests across many addresses so no single one sends enough traffic to look abusive. In Python you have two practical paths to do this. You can manage proxies yourself with a list and random selection, which is fine for a handful of static proxies you trust but quickly becomes a chore of health-checking and ban-handling. Or you can route everything through one rotating gateway that assigns a fresh IP per request on the server side, so your Python code never sees a proxy list at all. We will build up from the naive approach to the production-ready one.
Every example below sends requests through the gateway at gateway.proxyrotator.com:8080 over HTTPS, authenticating with USER:PASS. Your real host, port and credentials appear in your dashboard after signup, and you can choose your IP type and country there too. Prefer IP whitelisting? Add your server IP in the dashboard and drop the USER:PASS@ credentials from the URLs.
1. The naive way: a list of proxies and random.choice
The classic do-it-yourself approach keeps a Python list of proxy URLs and picks one at random for each request. It works, and it is worth understanding, but you own every downside: you must source and refresh the proxies, detect when one is dead or banned, and rotate around failures yourself.
import random
import requests
# You source and maintain this list yourself.
PROXIES = [
"http://USER:PASS@proxy1.example.com:8000",
"http://USER:PASS@proxy2.example.com:8000",
"http://USER:PASS@proxy3.example.com:8000",
]
def fetch(url):
proxy = random.choice(PROXIES)
proxies = {"http": proxy, "https": proxy}
return requests.get(url, proxies=proxies, timeout=20)
for _ in range(5):
try:
r = fetch("https://api.ipify.org")
print("IP:", r.text)
except requests.RequestException as e:
print("request failed:", e) # dead proxy? you handle it
The downsides are real. A random pick can land on the same proxy twice in a row, so spread is uneven. A dead or banned proxy raises an exception and you have to catch it, drop that proxy, and retry, none of which this snippet does well. And the list goes stale: proxies disappear and you are back to sourcing more. For anything beyond a quick test, the maintenance cost is the problem rotation was supposed to solve.
2. The easy way: one rotating gateway
Instead of a list, point every request at a single rotating gateway. The gateway assigns a new IP per request on its side, so your code holds no proxy list, does no health-checking, and writes no rotation logic. The proxies dict is identical for every call; the IP changes anyway.
import requests
# One endpoint. The gateway rotates the IP for you, per request.
GATEWAY = "https://USER:PASS@gateway.proxyrotator.com:8080"
proxies = {"http": GATEWAY, "https": GATEWAY}
for _ in range(5):
r = requests.get("https://api64.ipify.org", proxies=proxies, timeout=20)
print("IP:", r.text) # a different IP on each request
That is the whole rotation story. No list, no random.choice, no dead-proxy handling. Use https://api64.ipify.org if you want it to print IPv6 when you have selected the IPv6 type, or https://api.ipify.org for IPv4 only. To keep the same IP across a few requests instead of rotating every call, switch your plan to sticky mode in the dashboard; the code does not change. You can drop this gateway straight into your own tooling with the rotating proxy API.
3. Production-ready: retries and backoff
Networks are flaky and targets occasionally return 429 or 5xx. The robust pattern is a requests.Session with a urllib3 Retry mounted on an HTTPAdapter, so transient failures retry automatically with exponential backoff. Because you are on a rotating gateway, each retry naturally exits from a different IP, which often clears a temporary block on its own.
import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
GATEWAY = "https://USER:PASS@gateway.proxyrotator.com:8080"
def build_session():
retry = Retry(
total=5, # up to 5 attempts
backoff_factor=1.0, # 0s, 1s, 2s, 4s, 8s...
status_forcelist=(429, 500, 502, 503, 504),
allowed_methods=("GET", "POST"),
raise_on_status=False,
)
adapter = HTTPAdapter(max_retries=retry)
s = requests.Session()
s.mount("http://", adapter)
s.mount("https://", adapter)
s.proxies.update({"http": GATEWAY, "https": GATEWAY})
return s
session = build_session()
for _ in range(5):
r = session.get("https://api.ipify.org", timeout=20)
print(r.status_code, r.text)
Setting proxies on the session means every request reuses the gateway without repeating yourself. On older urllib3 versions use method_whitelist instead of allowed_methods. With raise_on_status=False the session returns the final response after exhausting retries rather than raising, so you can inspect the status code yourself.
4. Scrapy: set the proxy per request
Scrapy reads the proxy from request.meta["proxy"], and its built-in HttpProxyMiddleware applies it. The simplest approach is to set the gateway in meta when you yield each request, so every request routes through the rotating gateway.
import scrapy
GATEWAY = "https://USER:PASS@gateway.proxyrotator.com:8080"
class IpSpider(scrapy.Spider):
name = "ip"
start_urls = ["https://api.ipify.org"] * 5
def start_requests(self):
for url in self.start_urls:
yield scrapy.Request(
url,
meta={"proxy": GATEWAY},
dont_filter=True, # allow the same URL repeatedly
callback=self.parse,
)
def parse(self, response):
self.logger.info("IP: %s", response.text)
If you would rather not repeat meta in every request, set the proxy once in a small downloader middleware so all requests inherit it. The gateway still rotates the IP per request either way.
# middlewares.py
class RotatingProxyMiddleware:
GATEWAY = "https://USER:PASS@gateway.proxyrotator.com:8080"
def process_request(self, request, spider):
request.meta["proxy"] = self.GATEWAY
return None # let Scrapy continue with the proxy set
# settings.py
DOWNLOADER_MIDDLEWARES = {
"myproject.middlewares.RotatingProxyMiddleware": 610,
"scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware": 750,
}
RETRY_ENABLED = True
RETRY_TIMES = 5
RETRY_HTTP_CODES = [429, 500, 502, 503, 504]
Ordering matters: your middleware (610) runs before Scrapy's HttpProxyMiddleware (750), which reads the proxy you set and applies it. Scrapy's retry middleware then re-sends failed requests, and each retry exits from a fresh gateway IP. See the full Scrapy integration for more.
5. Async at scale: aiohttp
When you need many requests in flight at once, aiohttp sends them concurrently. Pass the gateway as the proxy argument on each request. Because aiohttp does not put credentials in the proxy URL the same way, supply them with aiohttp.BasicAuth via proxy_auth, and use an http:// scheme for the proxy endpoint.
import asyncio
import aiohttp
PROXY = "http://gateway.proxyrotator.com:8080"
PROXY_AUTH = aiohttp.BasicAuth("USER", "PASS")
async def fetch_ip(session):
async with session.get(
"https://api.ipify.org",
proxy=PROXY,
proxy_auth=PROXY_AUTH,
timeout=aiohttp.ClientTimeout(total=20),
) as resp:
return await resp.text()
async def main():
async with aiohttp.ClientSession() as session:
tasks = [fetch_ip(session) for _ in range(10)]
for ip in await asyncio.gather(*tasks):
print("IP:", ip) # concurrent requests, different IPs
asyncio.run(main())
Each of the ten concurrent requests routes through the gateway and exits from its own rotated IP, so a wide async crawl spreads across many addresses at once. Keep concurrency sensible and add your own retry wrapper around fetch_ip for production runs. For browser-driven work that needs JavaScript rendering, the same gateway plugs into our Selenium integration.
Confirming rotation is working
Whichever approach you use, the test is the same: call an IP-echo endpoint such as https://api.ipify.org several times and check that the returned address changes between calls. If it changes, rotation is live and your traffic is spreading across the pool. If it stays the same, you are likely in sticky mode, in which case switch to rotating in the dashboard, or your client may not be honouring the proxy, so double-check the proxies dict or proxy argument. From here, set your IP type and country in the dashboard and point the same gateway at your real targets. The gateway handles HTTPS and SOCKS5 and authenticates with user and password or an IP whitelist, so it slots into rotating proxies workflows of any size. For first-time setup, the how to set up rotating proxies guide walks through the dashboard step by step. Trusted since 2014 by more than 62,000 businesses.
Rotating IP in Python FAQ
How do I rotate my IP address in Python?
Do I need a list of proxies to rotate IPs in Python?
How do I set a proxy in Python requests?
How do I add retries and backoff when rotating IPs?
How do I rotate IPs in Scrapy?
How do I rotate IPs with aiohttp for async requests?
Why is my IP not changing between requests?
Can I choose a country or IP type when rotating in Python?
Does the rotating gateway support SOCKS5 in Python?
How much does a rotating proxy gateway cost?
Rotate IPs in Python without the proxy list
Point one gateway at requests, Scrapy or aiohttp and get a new IP per request from residential, datacenter, mobile and IPv6 pools. One plan, from $24.95/mo.