sherlock icon indicating copy to clipboard operation
sherlock copied to clipboard

Cloudscraper support + Better twitter URL.

Open darvell opened this issue 4 years ago • 0 comments

Adds cloudscraper as an optional import which will allow for some Cloudflare based sites to be checked. Non-mobile twitter links are now provided in the output.

Edit for extra notes:

Well, I'm not sure why the second test failed but that seems common. Cloudscraper works purely in Python (though it can use external JS engines if it wishes) and has the same API as Requests plus simulates a sane modern user agent (even if this is overrode). A lot of sites that have been removed because of bot checks are able to work better and I find myself having more consistent results as some websites appear to change in to 'you might be a bot' mode at random i n the current iteration of Sherlock.

darvell avatar Nov 08 '20 19:11 darvell