cloudflare-scrape icon indicating copy to clipboard operation
cloudflare-scrape copied to clipboard

Add urllib3 < 2.0.0 dependency

Open MatteoCampinoti94 opened this issue 2 years ago • 1 comments
trafficstars

urllib3 introduced a backwards incompatible change that removed the DEFAULT_CIPHERS field.

Causes the following error:

File "C:\Users\...\AppData\Local\pypoetry\Cache\virtualenvs\falr-05-2023-V98AjTAw-py3.10\lib\site-packages\faapi\connection.py", line 12, in <module>
    from cfscrape import CloudflareScraper  # type: ignore
  File "C:\Users\...\AppData\Local\pypoetry\Cache\virtualenvs\falr-05-2023-V98AjTAw-py3.10\lib\site-packages\cfscrape\__init__.py", line 19, in <module>
    from urllib3.util.ssl_ import create_urllib3_context, DEFAULT_CIPHERS
ImportError: cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_' (C:\Users\...\AppData\Local\pypoetry\Cache\virtualenvs\falr-05-2023-V98AjTAw-py3.10\lib\site-packages\urllib3\util\ssl_.py)

MatteoCampinoti94 avatar May 24 '23 13:05 MatteoCampinoti94

Please don't do that.

urllib3.util.ssl_ was just a binding to python's ssl lib. As the ssl's documentation mention it: DEFAULT_CIPHERS has been replaced by PROTOCOL_TLS_CLIENT in last versions.

Furthermore, taking the time to read the documentation, we could propose an up-to-date solution like:

  • cfscrape/__init__.py
import ssl

[...]

class CloudflareAdapter(HTTPAdapter):
    """ HTTPS adapter that creates a SSL context with custom ciphers """

    def get_connection(self, *args, **kwargs):
        context = ssl.SSLContext(protocol=ssl.PROTOCOL_TLS_CLIENT)
        context.check_hostname = False
        context.verify_mode = ssl.CERT_NONE
        conn.conn_kw["ssl_context"] = context

        return conn

Note: need some improvements! I just draft this solution from another patch I made to support last ssl version.

Pyvonix avatar May 31 '23 06:05 Pyvonix