cloudflare-scrape
cloudflare-scrape copied to clipboard
Unable to identify Cloudflare IUAM Javascript on website
Before creating an issue, first upgrade cfscrape with pip install -U cfscrape and see if you're still experiencing the problem. Please also confirm your Node version (node --version or nodejs --version) is version 10 or higher.
Make sure the website you're having issues with is actually using anti-bot protection by Cloudflare and not a competitor like Imperva Incapsula or Sucuri. And if you're using an anonymizing proxy, a VPN, or Tor, Cloudflare often flags those IPs and may block you or present you with a captcha as a result.
Please confirm the following statements and check the boxes before creating an issue:
- [x] I've upgraded cfscrape with
pip install -U cfscrape - [x] I'm using Node version 10 or higher
- [x] The site protection I'm having issues with is from Cloudflare
- [x] I'm not using Tor, a VPN, or an anonymizing proxy
Python version number:3.6.0
cfscrape version number:2.1.1
Code snippet involved with the issue
scraper = cfscrape.create_scraper(delay=5) print(scraper.get(session_url).content)
Complete exception and traceback
(If the problem doesn't involve an exception being raised, leave this blank)
ValueError: Unable to identify Cloudflare IUAM Javascript on website. Cloudflare may have changed their technique, or there may be a bug in the script.
URL of the Cloudflare-protected page: https://cn.investing.com/
[LINK GOES HERE]
URL of Pastebin/Gist with HTML source of protected page
[LINK GOES HERE]
I confirm:
- I've upgraded cfscrape with pip install -U cfscrape
- I'm using Node version 10 or higher
- The site protection I'm having issues with is from Cloudflare
- I'm not using Tor, a VPN, or an anonymizing proxy
Python version: 3.9.4 NodeJS: v14.8.0 cfscrape: 2.1.1
code snippet involving the issue:
import requests import cfscrape session = requests.Session() scraper = cfscrape.create_scraper(sess=session) res = scraper.get('https://www.discogs.com/artist/125246-Nirvana')
complete exception and traceback
`_--------------------------------------------------------------------------- AttributeError Traceback (most recent call last) ~\anaconda3\lib\site-packages\cfscrape_init_.py in solve_challenge(self, body, domain) 254 r"(?:[^{<>]},\s(\d{4,}))?", --> 255 javascript, flags=re.S 256 ).groups()
AttributeError: 'NoneType' object has no attribute 'groups'
During handling of the above exception, another exception occurred:
ValueError Traceback (most recent call last)
~\anaconda3\lib\site-packages\requests\sessions.py in get(self, url, **kwargs) 544 545 kwargs.setdefault('allow_redirects', True) --> 546 return self.request('GET', url, **kwargs) 547 548 def options(self, url, **kwargs):
~\anaconda3\lib\site-packages\cfscrape_init_.py in request(self, method, url, *args, **kwargs) 127 # Check if Cloudflare anti-bot "I'm Under Attack Mode" is enabled 128 if self.is_cloudflare_iuam_challenge(resp): --> 129 resp = self.solve_cf_challenge(resp, **kwargs) 130 131 return resp
~\anaconda3\lib\site-packages\cfscrape_init_.py in solve_cf_challenge(self, resp, **original_kwargs) 202 203 # Solve the Javascript challenge --> 204 answer, delay = self.solve_challenge(body, domain) 205 if method == 'POST': 206 cloudflare_kwargs["data"]["jschl_answer"] = answer
~\anaconda3\lib\site-packages\cfscrape_init_.py in solve_challenge(self, body, domain) 290 raise ValueError( 291 "Unable to identify Cloudflare IUAM Javascript on website. %s" --> 292 % BUG_REPORT 293 ) 294
ValueError: Unable to identify Cloudflare IUAM Javascript on website. Cloudflare may have changed their technique, or there may be a bug in the script.
Please read https://github.com/Anorov/cloudflare-scrape#updates, then file a bug report at https://github.com/Anorov/cloudflare-scrape/issues."_`
This library is unmaintained and dead, see #406
This library is unmaintained and dead, see #406
Thanks for your response. I have noticed that issue only after my comment.
This library is unmaintained and dead, see #406
Also https://github.com/VeNoMouS/cloudscraper/ just republish but script still not work
This library is unmaintained and dead, see #406
Also https://github.com/VeNoMouS/cloudscraper/ just republish but script still not work
I just saw on twitter that 1 week ago, cloudflare posted an announcement about '' cloudscraper '' saying that the new update destroyed cloudscraper forever I think that the cloudscraper admin @VeNoMouS can no longer fix this public repo, probably only the version paid by him will keep it updated
a new method would be to use puppeteer, I found this version of cloudscraper based on puppeteer https://github.com/ryxnSZN/cloudscraper it needs a few updates to bypass the IUAM page
Cloudflare no longer uses the cfduid cookie in headers (set-cookie)
// https://github.com/scaredos/cfresearch here some info about new challenge
I choose not to update the public one, its not a case of "can no longer fix"...
The paid subscription code works fine... (find me on discord if you wish to inquire)