aiohttp proxy errors: ClientHttpProxyError, SOCKS5 and the environment
aiohttp tells you exactly what the proxy said, and your password along with it. Each error we hit, what it means, and a setup that keeps both under control.
In short
ClientHttpProxyError carries the proxy’s status in err.status: 407 the login, 403 a username value our gateway can’t read, 465 no exit for your filters. Log the status, not the message: it contains the proxy URL with the password. For SOCKS5 you need aiohttp-socks.
Every error, reproduced
One mistake at a time, against our gateway, https:// target unless noted.
| What we broke | What aiohttp raised |
|---|---|
| Wrong password, or no login | ClientHttpProxyError: 407, message='Proxy Authentication Required' |
| A username value the proxy can’t read (-country-usa) | ClientHttpProxyError: 403, message='Invalid Request' |
| Filters no exit matches (Chicago in Germany) | ClientHttpProxyError: 465, message='Can Not Fulfill Targeting' |
| Wrong password, http:// target | no exception: response.status == 407 |
| An unescaped @ in the password | 407, the @ was encoded as %40 and the password no longer matched |
| https:// as the proxy scheme | ClientConnectorSSLError … WRONG_VERSION_NUMBER |
| socks5:// as the proxy scheme | ClientResponseError: 400, Bad status line … b'\x01\x01' |
407 and the escaped password
A wrong or missing login is 407. One less obvious cause: a password with @ written into the URL as it is. aiohttp didn’t fail to parse it; it encoded the @ as %40 and sent a different password. Build the URL with urllib.parse.quote(value, safe="") for both parts.
403 and 465: the username’s targeting
Our gateway reads country, city, network and session from the username. A value it can’t read is 403; a filter no residential exit matches right now is 465. A key it doesn’t know, such as -cc-DE, is ignored without an error. The same codes in other clients: requests, Scrapy.
Look at what str(err) contains for a refused tunnel. aiohttp appends the proxy URL, login included:
aiohttp.client_exceptions.ClientHttpProxyError: 465, message='Can Not Fulfill Targeting', url='http://USER-country-de-city-chicago:PASS@via.proxies.vision:9000'A scraper that logs exceptions, or reports them to an error tracker, writes your proxy password there with every refusal. Catch ClientHttpProxyError and log err.status instead, as the setup below does.
socks5:// needs aiohttp-socks
aiohttp has no SOCKS support, and it doesn’t reject a socks5:// URL either: it spoke HTTP to the SOCKS port and failed on the reply, Bad status line … b'\x01\x01'. Install aiohttp-socks and pass ProxyConnector.from_url("socks5://USER:PASS@via.proxies.vision:11000", rdns=True) as the session’s connector; that returned a German exit in our test. For plain web requests the HTTP port is simpler.
The environment is ignored by default
Unlike requests, aiohttp doesn’t read HTTPS_PROXY unless you ask:
proxy_auth is on its way out
proxy_auth=aiohttp.BasicAuth(…) still worked in 3.14, with two DeprecationWarnings: both the parameter and BasicAuth are to be removed in aiohttp 4.0. The login in the proxy URL works on every version.
Our rotating port picks an exit per connection, and a ClientSession reuses its connections:
For the same IP on purpose, use a session label on the sticky port: USER-session-checkout1-lifetime-10 on via.proxies.vision:10000.
A setup that works
An escaped login, a new exit per request, the status logged without the password, retries only where they help. Ran as shown.
import asyncio
import os
from urllib.parse import quote
import aiohttp
USER = os.environ["PROXY_USER"]
PASS = os.environ["PROXY_PASS"]
def proxy_url(targeting: str = "") -> str:
# quote() keeps @ : / in the password from breaking the URL.
return f"http://{quote(USER + targeting, safe='')}:{quote(PASS, safe='')}@via.proxies.vision:9000"
async def fetch(url: str, targeting: str = "", attempts: int = 3) -> str:
for attempt in range(1, attempts + 1):
# force_close: a new connection per request, so a new exit IP.
connector = aiohttp.TCPConnector(force_close=True)
try:
async with aiohttp.ClientSession(connector=connector, timeout=aiohttp.ClientTimeout(total=30)) as session:
async with session.get(url, proxy=proxy_url(targeting)) as response:
return await response.text()
except aiohttp.ClientHttpProxyError as err:
# err.status is the proxy's answer. Don't log str(err): it contains the proxy URL, password included.
if err.status in (403, 407) or attempt == attempts:
raise RuntimeError(f"proxy refused with {err.status}") from None
except aiohttp.ClientError:
if attempt == attempts:
raise # a dropped exit, three times in a row
async def main() -> None:
ips = await asyncio.gather(*[fetch("https://api.ipify.org", "-country-de") for _ in range(5)])
print(ips)
try:
await fetch("https://api.ipify.org", "-country-usa")
except RuntimeError as err:
print(err)
asyncio.run(main())['80.144.102.131', '93.195.66.228', '85.16.88.131', '79.207.68.246', '31.17.85.19']
proxy refused with 403Why only some errors are retried
A residential exit can drop a connection; a new one gets a new exit, so other ClientErrors are retried. A 465 may clear when another exit comes online. 403 and 407 are about your settings and fail the same way every time.
What does aiohttp.ClientHttpProxyError: 407 mean?
Does aiohttp support SOCKS5 proxies?
Does aiohttp use HTTP_PROXY or HTTPS_PROXY?
Is proxy_auth deprecated in aiohttp?
Why does aiohttp keep the same proxy IP?
Keep reading
Missing dependencies for SOCKS support: the Fix, and Why You See It Without SOCKS
Install requests[socks], or find the environment variable that asked for SOCKS. Plus socks5 vs socks5h and the errors that come next.
6 min read · #python#socks5
curl: (56) CONNECT tunnel failed, response 407 — and git's Proxy CONNECT aborted
What curl's 'CONNECT tunnel failed, response 407' means, every curl proxy exit code we hit, and why git fails through a proxy that works in curl.
6 min read · #errors#authentication
Playwright Proxy Errors: ERR_TUNNEL_CONNECTION_FAILED, NS_ERROR_PROXY_*
Every Playwright proxy error in Chromium and Firefox, what the proxy answered, the silent 30-second timeout on a wrong password, and SOCKS5 auth.
7 min read · #playwright#errors
Residential proxies, pay per GB
HTTP and SOCKS5. Country, city and sticky sessions in the username. No subscription.