scrapy-cloudflare-middleware
scrapy-cloudflare-middleware copied to clipboard
A Scrapy middleware to bypass the CloudFlare's anti-bot protection
Hi @clemfromspace can I take ownership of this project? I currently use cloudscraper library with scrapy to pass cloudflare protection, but it is not a middleware as you know and...
It seems that your code snippet didn't handle the requests already in the scheduler queue while the concurrent one request challenged by cloudflare. Therefore even if a challenge is resolved,...
Changed the paths in set up imports given changes in pip
> $ pip install scrapy_cloudflare_middleware ``` Collecting scrapy_cloudflare_middleware Using cached https://files.pythonhosted.org/packages/40/a9/318f4d04af28560ec676989ba419d4e43e5908a90fd5bd6a236b9ec5e178/scrapy_cloudflare_middleware-0.0.1.tar.gz Complete output from command python setup.py egg_info: Traceback (most recent call last): File "", line 1, in File "/tmp/pip-install-I2lEg1/scrapy-cloudflare-middleware/setup.py",...