broken-links-checker
broken-links-checker copied to clipboard
Usage
Configure urls to parse in config.py. Then, run:
scrapy runspider broken_links_spider.py -o output.json
Then check 404 items in the output.json file.