Eugene Yurkevich

Hi guys, is there any way to detect `bot` (`crawler`) by User-Agent in this library? ``` $mobileDetect->is('bot'); $mobileDetect->is('Bot'); $mobileDetect->is('mobileBot'); $mobileDetect->is('MobileBot'); ``` It seems that is all is `false` using latest...

We can Log message. The expected behavior is that we skip directives with typos? ``` User-Agent: * Disallow: /path1 Disalow: /path2 ``` expected to disallow only /path1?

But are you sure that it is a good idea to add typos logs to this repo? I asked because it will be more code, more logic, which can be...

Yes, you are right @scott8035 . The initial intention was as in code you provided. In general it should work, but extra condition does not give profit. Will have a...

At the moment > Disallow: /Example/* and > Disallow: /Example/ Will be transform into regexp > /^\/Example\/.*/ and > /^\/Example\// which will equally work eventually (the end of a string)....

`^/Example` `^/Example.*` These both regexps will work identical. We can do it, but it will not have effect. We can log message for user information. Also here https://github.com/bopoda/robots-txt-parser/issues/9.

@LeMoussel Do you mean that we should skip rule if path starts not from "/" ?

If yes, i agree with you. But parser can return rules "as is". I have plans to add new class `new RobotsTxtValidator(array $rules /*rules from robotsTxtParser*/)` (or may be upgrade...

possible, but i`m not sure that rules without '/' at the beginning are valid case for search robots. Need investigate what google/yandex do with that urls :)

http://joxi.ru/KAg7BEeC4e8EdA http://joxi.ru/bmo6JBLSxoJl1r As result of my investigation, we can skip all rules in **allow** and **disallow** directives except those which begin with `/` or `*`. Seems it does not cause...