Crawler-Detect
Crawler-Detect copied to clipboard
JayBizzle
🕷 CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent
Hello guys. First of all, thank you for your package. It is really helpful for projects I've been doing. Literally, I'm referencing existing issue #309 but from a little different...
This is not a firm decision, just exploring the possibility only supporting officially supported PHP versions - https://www.php.net/supported-versions.php
There are EXACTLY like this user agents: ```Anthill``` and ```facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)``` 
Need to look into this... https://web.dev/user-agent-client-hints/
Hi, I've install Crawler-Detect, it work fine but urlscan.io can crawl. For now, I still have not succeeded in obtaining their fingerprints but I prefer to warn, maybe you will....
Lines extracts from a few websites in Russia with high traffic. Each suggested line [checked](https://crawlerdetect.io/?q=) with «User agent is NOT a bot» result. [Sample sources](https://gist.github.com/antonydevanchi/d673f277d6087326bdb46e03dac37595). @JayBizzle please, approve list and...
- [x] Filestack - [x] Google-Ads-Overview Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36 - [x] Google-Ads-Overview Mozilla/5.0 (Linux; U; Android 6.0.1; generic) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118...
I have found some new possibilities, but I not right what should be added or not, or even if it is really a crawler. So I am creating this issue...
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36 ForusP ForusP is a security company and uses Qualys to perform automated tests on websites. Above useragent they...
