cloudflare-scrape
cloudflare-scrape copied to clipboard
Fails on site
[ x] I've upgraded cfscrape with pip install -U cfscrape [ x] I'm using Node version 10 or higher [ x] The site protection I'm having issues with is from Cloudflare
Centos 7 x64, Python 3.6.8, cfscrape==2.1.1, node v10.20.1 import cfscrape scraper = cfscrape.create_scraper() scraper.get("https://arma-models.ru").content
Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/scrapy/crawler.py", line 184, in crawl return self._crawl(crawler, *args, **kwargs) File "/usr/lib/python3.6/site-packages/scrapy/crawler.py", line 188, in _crawl d = crawler.crawl(*args, **kwargs) File "/usr/lib64/python3.6/site-packages/twisted/internet/defer.py", line 1613, in unwindGenerator return _cancellableInlineCallbacks(gen) File "/usr/lib64/python3.6/site-packages/twisted/internet/defer.py", line 1529, in _cancellableInlineCallbacks _inlineCallbacks(None, g, status) ---
--- File "/usr/lib64/python3.6/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks result = g.send(result) File "/usr/lib/python3.6/site-packages/scrapy/crawler.py", line 87, in crawl start_requests = iter(self.spider.start_requests()) File "/var/python_script/model-lavka.ru/model_lavka/model_lavka/spiders/armata_exper.py", line 44, in start_requests print(cfscrape.get_tokens(self.start_urls[0], self.user_agent)) File "/usr/lib/python3.6/site-packages/cfscrape/init.py", line 383, in get_tokens resp = scraper.get(url, **kwargs) File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 537, in get return self.request('GET', url, **kwargs) File "/usr/lib/python3.6/site-packages/cfscrape/init.py", line 129, in request resp = self.solve_cf_challenge(resp, **kwargs) File "/usr/lib/python3.6/site-packages/cfscrape/init.py", line 204, in solve_cf_challenge answer, delay = self.solve_challenge(body, domain) File "/usr/lib/python3.6/site-packages/cfscrape/init.py", line 292, in solve_challenge % BUG_REPORT builtins.ValueError: Unable to identify Cloudflare IUAM Javascript on website. Cloudflare may have changed their technique, or there may be a bug in the script. Please read https://github.com/Anorov/cloudflare-scrape#updates, then file a bug report at https://github.com/Anorov/cloudflare-scrape/issues."
2020-05-02 01:37:31 [twisted] CRITICAL: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/cfscrape/init.py", line 255, in solve_challenge javascript, flags=re.S AttributeError: 'NoneType' object has no attribute 'groups'
During handling of the above exception, another exception occurred: