#Scrapy
Proxy middleware and rotation in Scrapy.
2 articles
Fixes
Scrapy: Could not open CONNECT tunnel with proxy (407, 403, SOCKS5)
What the status in Scrapy's TunnelError means, why socks5:// only times out, and a proxy middleware that works. Every case reproduced.
7 min read · #scrapy#python
Setup guides
How to Rotate Proxies in Python (and Why Yours May Not Rotate)
requests.Session, httpx.Client and Scrapy reuse one IP through a rotating proxy. What rotates, measured, and code that gets a new IP per request.
7 min read · #python#scrapy