How to Use Proxies to Rotate IP Addresses in Python
Published on Aug. 22, 2023, 12:15 p.m.
To use proxies to rotate IP addresses in Python, you can use the requests
library along with a list of proxies. Here is an example code that shows how to send an HTTP GET request using a rotating proxy:
import requests
from itertools import cycle
# define the list of proxies
proxies = [
'http://proxy1.com:1234',
'http://proxy2.com:2345',
'http://proxy3.com:3456'
]
# create an iterator that will loop over the proxies
proxy_pool = cycle(proxies)
url = 'https://www.example.com/'
# send GET requests using rotating proxies
for i in range(3):
# get the next proxy from the proxy pool
proxy = next(proxy_pool)
print(f"Using proxy {proxy}")
try:
response = requests.get(url, proxies={'http': proxy, 'https': proxy}, timeout=5.0)
print(response.status_code)
print(response.content)
except Exception as e:
print(e)
In this example, we have defined a list of proxies with their respective URLs. We then create an iterator called proxy_pool
that will loop over the proxies indefinitely using the cycle()
function imported from itertools.
We then send GET requests to a target URL using the requests.get()
method, and we pass a dictionary of proxies along with the request using the proxies
parameter. The timeout
parameter is set to 5.0
to avoid waiting indefinitely for a response.
We then use a try-except block to catch any exceptions that may arise from sending the requests, such as connection timeouts or invalid proxies.