hakrawler
hakrawler copied to clipboard
Add `-i` flag (crawl inside path) and display an error on stderr if improper URLs are being given
url.Parse
does not return an error most of the time because it treats random strings as relative URLs.
Just pushed another update that adds the -i
flag for limiting the crawl to inside a path.
for example echo https://example.com/outer_path/path1 | hakrawler -i
will not crawl https://example.com/outer_path/
This is really great work, thanks garlic0x1 🙂