cfcrawler icon indicating copy to clipboard operation
cfcrawler copied to clipboard

Cloudflare scraper and cralwer written in Async, In-place library for HTTPX. Crawl website that has cloudflare enabled, easier than ever!

Results 3 cfcrawler issues
Sort by recently updated
recently updated
newest added

I get caught by Cloudflare when I use a proxy. Proxies are clean. There is no proxy-related blocking, I can connect with another library. ` proxies = { 'http://': 'http://username:password',...

```python from cfcrawler import AsyncClient ``` ```python-traceback Traceback (most recent call last): File "", line 1, in from cfcrawler import AsyncClient File "/Users/bob/cfcrawler-venv/lib/python3.13/site-packages/cfcrawler/__init__.py", line 1, in from cfcrawler.client import AsyncClient...

``` from cfcrawler import AsyncClient client = AsyncClient(use_fake_useragent_library=True) client.headers # Headers({'accept': '*/*', 'accept-encoding': 'gzip, deflate, br', 'connection': 'keep-alive', 'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0...