cfscrape.get(url).content returns "Attention Required! | Cloudflare" page

[JafarAbbas33]

trafficstars

Please confirm the following statements and check the boxes before creating an issue:

• [X ] I've upgraded cfscrape with pip install -U cfscrape
• [ ] I'm using Node version 10 or higher
• [X ] The site protection I'm having issues with is from Cloudflare
• [X ] I'm not using Tor, a VPN, or an anonymizing proxy

Python version number

Run python --version and paste the output below: -> Python 3.8.2

cfscrape version number

Run pip show cfscrape and paste the output below: -> Name: cfscrape Version: 2.1.1 Summary: A simple Python module to bypass Cloudflare's anti-bot page. See https://github.com/Anorov/cloudflare-scrape for more information. Home-page: https://github.com/Anorov/cloudflare-scrape Author: Anorov Author-email: [email protected] License: UNKNOWN Location: c:\users\jafar33\appdata\local\programs\python\python38-32\lib\site-packages Requires: requests Required-by:

Code snippet involved with the issue

scraper = cfscrape.create_scraper()
url = "https://www.pexels.com/search/man/?format=js&seed=&page=2&type="
htmltext = scraper.get(url).content

URL of the Cloudflare-protected page

https://www.pexels.com/search/man/?format=js&seed=&page=2&type=

[caipenpen]

Note: This only works when regular Cloudflare anti-bots is enabled (the "Checking your browser before accessing..." loading page). If there is a reCAPTCHA challenge, you're out of luck. Thankfully, the JavaScript check page is much more common.

[JafarAbbas33]

But if you go to Pexels then you can see there is no captcha required. Then why is that happening? (Note: I have previously used cloudscraper which used to work but stopped working recently so I thought maybe i should move on to another library.)

[andress134]

something know any lib that can bypass new captcha challenge?

[JafarAbbas33]

@andress134 You having the same problem?

[andress134]

@andress134 You having the same problem?

Yes, cfscrape can't bypass captcha, just iuam v1 challenge. On this moment i think nobody (public lib) like cloudscraper, no one can't bypass new challenge

[caipenpen]

But if you go to Pexels then you can see there is no captcha required. Then why is that happening? (Note: I have previously used cloudscraper which used to work but stopped working recently so I thought maybe i should move on to another library.)

No , it have . Yesterday , I check and it need me resolve captcha ......... Please try again today , I see it already close the captcha

[JafarAbbas33]

I don't have to solve a captcha. I think it is because of some cookie. I took a snapshot of cookies used by the session but none of them made any sense.

[caipenpen]

I don't have to solve a captcha. I think it is because of some cookie. I took a snapshot of cookies used by the session but none of them made any sense.

What do you need to do this website ? Ddos it ???

[JafarAbbas33]

@caipenpen Cmon, no man XD. I want to automate getting some photos from a search result.

[andress134]

Looking for some user who can make a captcha bypass using public lib like 2captcha, anticaptcha or who can fix cloudscraper(nodejs) captcha function I can pay 200$

[caipenpen]

@caipenpen Cmon, no man XD. I want to automate getting some photos from a search result.

Why don't you use API ?

[JafarAbbas33]

@caipenpen To be truthful, I have no idea why that didn't occur to me. Thanks a lot! But still this library needs to be fixed. Right?

[caipenpen]

@caipenpen To be truthful, I have no idea why that didn't occur to me. Thanks a lot! But still this library needs to be fixed. Right?

Yes , but we use it free and the cloudflare is update day by day , we can't rush the developer . I think you should try use api of pexels , I see it have search option

[JafarAbbas33]

Yes you are right. And thanks to you, I can again fetch photos from Pexels (ofcourse I tried and it worked like a charm). So I will leave this issue opened as it has to be solved anyway. Thanks everyone.

[F170x]

I have the same issue with another site, when you open the url from your browser (chrome, firefox, safari, etc) works perfect without captcha required. Also in incognit mode. But when you try to make a request with this library I get the captcha error.

I think that cloudflare maybe detect anything in the headers (sort of headers, upper or lower cases, etc) or in the ssl protocol (TLS version, cypher, etc)

Anyone have updates about this?