ProxyBroker icon indicating copy to clipboard operation
ProxyBroker copied to clipboard

crawler 550 proxy, and the process hangs(pending).

Open zplzpl opened this issue 5 years ago • 2 comments

proxybroker find --types HTTP HTTPS --lvl High

Running this command, I only collected 550 proxy IPs, then the process does not work and does not exit.

is this normal?

zplzpl avatar Aug 09 '19 05:08 zplzpl

When creating an issue, please post at least a minimum required info.

OS: Windows 10 Terminal: Git Bash I tried it rigth now and yes, i got around 621 and the process keept hanging.

OS: Windows 10 Terminal: Cmd Same thing here, i've got around 605 proxies and the process just keept hanging.

So, is this normal? Seems at least common. Is that good? No, sure it is not. I've not digged on what is causing this, but I don't remember if this happend to me in the past, because i always supplied --limit, or even better, used the option serve and using the middle proxy directly instead of keep files.

You could also just --limit 500. It may appear that never ends it but will do. I don't really understand what's getting wrong here. Async tasks should be cancelled after --limit is reached and I didn't checked the code to ensure that is doing it. Maybe the developer can tell you what's wrong :-(

I'll be back later when i test on Linux.

erm3nda avatar Dec 20 '19 02:12 erm3nda

same question +1

gMan1990 avatar May 27 '20 02:05 gMan1990