cfcrawler
cfcrawler copied to clipboard
[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] with google.com, youtube.com
from cfcrawler import AsyncClient
client = AsyncClient(use_fake_useragent_library=True)
client.headers
# Headers({'accept': '*/*', 'accept-encoding': 'gzip, deflate, br', 'connection': 'keep-alive', 'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 Edg/122.0.0.0'})
await client.get("https://youtube.com")
# ConnectError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] ssl/tls alert handshake failure
(_ssl.c:1000)
> pip freeze | grep "http\|cfc"
aiohttp==3.10.10
cfcrawler==0.1.0
httpcore==1.0.6
httpx==0.27.2
For anyone interested - this project's approach on customizing ssl.SSLContext not only can't achieve JA3 fingerprints compliance (and therefore - pass current Cloudflare bot detection) with selected user-agent (especially randomly-selected!) - because in the end OpenSSL is used instead of native browser's BoringSSL /nss. It also doesn't simulate HTTP/2.
I think this notice and links below should be added to README - today this approach isn't working with cloudflare at all.
For a solution take a look into curl-impersonate, python binding curl-cffi, httpx adapter for curl-cffi