linkcheck icon indicating copy to clipboard operation
linkcheck copied to clipboard

Published 20 hours ago • verified publisher icon filiph

Proposal to integrate the `robots_txt` package.

Open vxern opened this issue 2 years ago • 3 comments

Hi, author of the robots_txt package here. I thought that, since linkcheck currently features its own implementation of a parser for robots.txt, it would be a good idea to instead depend on something more complete, tested and documented.

vxern avatar Jan 09 '23 17:01 vxern

Thanks for the suggestion, @vxern! I'm afraid I don't have the time to implement this. Are you suggesting you'd implement this as well?

Also, cc @parlough for his opinion.

filiph avatar Jan 09 '23 17:01 filiph

Hi, @filiph. Sure, I wouldn't mind. Let's wait for parlough's opinion first, though. :)

vxern avatar Jan 09 '23 22:01 vxern

Sounds good to me, but it's a bit hard to determine for sure without seeing what the end result would look like.

I'd be happy to take a look if you open a PR :)

parlough avatar Jan 10 '23 00:01 parlough