crawlee icon indicating copy to clipboard operation
crawlee copied to clipboard

Add pendingRequestCount field to BasicCrawler

Open jancurn opened this issue 6 years ago • 1 comments

This will be similar to handledRequestsCount, but it will indicate how many requests are yet to be processed. The users of BasicCrawler can then use this field to determine whether they need to enqueue more requests during recursive crawling (there's no point to do it if you know there are too many already).

jancurn avatar Jul 02 '19 12:07 jancurn

Related: https://github.com/apifytech/apify-js/issues/321

mnmkng avatar Jul 03 '19 09:07 mnmkng