crawlee-python
crawlee-python copied to clipboard
Implement purging request queue in repeated `BasicCrawler.run` calls
See https://github.com/apify/crawlee/blob/2d5d443da5fa701b21aec003d4d84797882bc175/packages/basic-crawler/src/internals/basic-crawler.ts#L836-L845 for inspiration