cfcrawler icon indicating copy to clipboard operation
cfcrawler copied to clipboard

I can't bypass Cloudflare when I use a proxy

Open tufkubie opened this issue 1 year ago • 1 comments

I get caught by Cloudflare when I use a proxy. Proxies are clean. There is no proxy-related blocking, I can connect with another library.

proxies = { 'http://': 'http://username:password', 'https://': 'http://username:password', } async with AsyncClient(proxies=proxies) as client:

tufkubie avatar Sep 09 '24 02:09 tufkubie

Hi, I can't seem to reproduce your issue. can you please tell me which website you're crawling?

ManiMozaffar avatar Sep 19 '24 15:09 ManiMozaffar

closing this issue due to lack of activity and cannot reproduce

ManiMozaffar avatar Oct 24 '24 09:10 ManiMozaffar

@ManiMozaffar Hi, i got the same issue.

I am using residential proxies from StormProxies, not problem when using request.get And i am scraping images from https://www.invaluable.com/ ( example : https://image.invaluable.com/housePhotos/Dusseldorfer/22/781322/H0461-L387459107.jpg )

How can we try to solve this issue ?

I have no problem when i doesn't use proxies.

djodjo02130 avatar Oct 30 '24 00:10 djodjo02130