botasaurus icon indicating copy to clipboard operation
botasaurus copied to clipboard

Depth limit to set how levels deep links to be crawled.

Open sonupandey228 opened this issue 1 year ago • 0 comments

Is there any way to set the depth limit of links to be crawled ? Similar feature present in scrapy that controls how many levels deep the spider will go when following links from the initial URLs provided.

sonupandey228 avatar Jun 25 '24 07:06 sonupandey228