Damn-Small-URL-Crawler
Damn-Small-URL-Crawler copied to clipboard
Deep Crawl
Your docs say:
DeepCrawl and Show Fuzzable Links python3 dsuc.py -u -d http://testsite.com -f
But it doesn't work. This does work though:
DeepCrawl and Show Fuzzable Links python3 dsuc.py -d -u http://testsite.com -f
It certainly pulls in more URLs than the basic one, but still don't think it is grabbing all of the URLs for my domain. How deep is DeepCrawl supposed to be?
@mgifford looks like it goes 2 links deep, according to the doc.
Ok, so there's really no deep crawl if it is a question of going 1 layer or 2 layers down.