Steven

Thanks for reporting this. I agree that Abot may not be the best initializer of this setting. I will leave this issue open so others have a workaround.

agreed @alex-jitbit , changed the subject for better phrase indexing

Would love a pull request if you can craft one up without breaking any contracts or tests.

I'm having the same issue... See related [github issue here](https://github.com/sjdirect/abotx/issues/1) <Expiration>Fri, 31 Dec 9999 23:59:59 GMT</Expiration> This date blows up on just some of my customer's machines. I have not...

Looks like when you have a license with a expiration date of DateTime.MaxValue which is (Fri, 31 Dec 9999 23:59:59 GMT), some version of visual studio/.net fail. I was unable...

Hi appreciate circling back to share your solution. It does seem like a reasonable approach. I don't think it's something we should add back to the main solution but lets...

Looks like NRobots is converting the "Disallow: /?/" into "Disallow: /". The last checkin to that lib fixed a similar issue but obviously this presents other issues.

Even though this bug still exists in the nrobots lib, abot gives a workaround to this and similar issues with robots.txt preventing the crawl. See https://github.com/sjdirect/abot/commit/9bd3d7d91ebefb6e03ee2c2a1b5140cc4020073c for details. There is...

Moved from https://github.com/sjdirect/abot/issues/186