disallow topic
List
disallow repositories
RobotsValidator
21
Stars
2
Forks
Watchers
A python script to check if URLs are allowed or disallowed by a robots.txt file.
