icrawler

icrawler copied to clipboard

Parse / Donwloader stuck in inf loop if it did not reach max_num

7

When using the crawler I've encountered with issue regarding the max_num. If cases where less images found than the provided "max_num", a infinite loop begin, stop any previous result from...

zMynxx

Duplicate URLs?

1

Hello! I was reading the implementation and got a bit surprised in finding that there seems to be no check for duplicate URLs before adding them to the queue. Is...

MollyDream