--parserobots doesn't seem to work

[varna9000]

If I use the --parserobots argument the crawler simply crawls the main domain but doesn't follow any links dispite the allow directives.

You can try it on https://google.com to see the behaviour.