cloudflare-scrape
cloudflare-scrape copied to clipboard
Add urllib3 < 2.0.0 dependency
trafficstars
urllib3 introduced a backwards incompatible change that removed the DEFAULT_CIPHERS field.
Causes the following error:
File "C:\Users\...\AppData\Local\pypoetry\Cache\virtualenvs\falr-05-2023-V98AjTAw-py3.10\lib\site-packages\faapi\connection.py", line 12, in <module>
from cfscrape import CloudflareScraper # type: ignore
File "C:\Users\...\AppData\Local\pypoetry\Cache\virtualenvs\falr-05-2023-V98AjTAw-py3.10\lib\site-packages\cfscrape\__init__.py", line 19, in <module>
from urllib3.util.ssl_ import create_urllib3_context, DEFAULT_CIPHERS
ImportError: cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_' (C:\Users\...\AppData\Local\pypoetry\Cache\virtualenvs\falr-05-2023-V98AjTAw-py3.10\lib\site-packages\urllib3\util\ssl_.py)
Please don't do that.
urllib3.util.ssl_ was just a binding to python's ssl lib.
As the ssl's documentation mention it: DEFAULT_CIPHERS has been replaced by PROTOCOL_TLS_CLIENT in last versions.
Furthermore, taking the time to read the documentation, we could propose an up-to-date solution like:
cfscrape/__init__.py
import ssl
[...]
class CloudflareAdapter(HTTPAdapter):
""" HTTPS adapter that creates a SSL context with custom ciphers """
def get_connection(self, *args, **kwargs):
context = ssl.SSLContext(protocol=ssl.PROTOCOL_TLS_CLIENT)
context.check_hostname = False
context.verify_mode = ssl.CERT_NONE
conn.conn_kw["ssl_context"] = context
return conn
Note: need some improvements! I just draft this solution from another patch I made to support last ssl version.