AI crawler
Residential proxies in Crawl4AI
Crawl4AI turns pages into clean Markdown for LLMs. It sets proxies per request through CrawlerRunConfig, so each crawl can leave from a different country or keep one home IP for a session.
- Host
- via.proxies.vision
- Ports
- 9000 rotating · 10000 sticky
- Same IP for
- up to 24 hours
- Price
- $1/GB, never expire
The code
Crawl4AI wants the proxy on CrawlerRunConfig; the proxy argument on BrowserConfig is deprecated. Replace USER and PASS with your login from the dashboard.
import asyncio
from crawl4ai import AsyncWebCrawler, BrowserConfig, CrawlerRunConfig, ProxyConfig
run_config = CrawlerRunConfig(
proxy_config=ProxyConfig(
server="http://via.proxies.vision:9000",
username="USER-country-de",
password="PASS",
)
)
async def main():
async with AsyncWebCrawler(config=BrowserConfig()) as crawler:
result = await crawler.arun(url="https://example.com", config=run_config)
print(result.success, result.url)
asyncio.run(main())Set it up in Crawl4AI
As in Crawl4AI's documentation today.
01
Get your login
Sign up and copy your proxy username and password from the dashboard.
02
Build a ProxyConfig
Server http://via.proxies.vision:9000, your username with the country in it, your password.
03
Set it per request
Pass it as proxy_config on CrawlerRunConfig. A plain dict with the same three keys works too.
04
Sticky when you need it
For a crawl behind a login, add a session to the username and switch to port 10000 so every page leaves from the same IP.
Questions
- Rotating or sticky?
- Port 9000 gives every new connection a new home IP, good for crawling many pages. For a logged-in flow that has to keep one IP, put a session in the username (USER-session-job1-lifetime-30) and use port 10000; the same session name keeps the same exit for its lifetime.
- Can different requests use different countries?
- Yes. Each CrawlerRunConfig carries its own proxy_config, so build one per country (USER-country-de, USER-country-fr) and pass the one you need.
- What does it cost?
- $1 per GB of traffic, whatever the country, city or network. No subscription, and unused bandwidth never expires. An agent that loads full pages uses more than one fetching HTML, so block images and media where the task allows it.
