Kotsu icon indicating copy to clipboard operation
Kotsu copied to clipboard

Read disallow from robots.txt and pass to `sitemap_xml` ignores

Open ArmorDarks opened this issue 7 years ago • 0 comments

Well, it is slightly nontrivial, but would be cool to prevent some nasty situation when you ignored something in robots.txt, but not in sitemap_xml task

ArmorDarks avatar Jul 27 '17 21:07 ArmorDarks