ProxyBroker
ProxyBroker copied to clipboard
constverum
rotation doesn't work
$ proxybroker serve --host 127.0.0.1 --port 8888 --types HTTP HTTPS --lvl High
This seems to only ever use one proxy.
Totally yes, and seems reusing them even if relaunch it. The only way i saw to get another is to change --strict or --types or whatever other parameter.
I'm not using this since years, but as far as i remember, it rotated proxies in the past. Right now i don't know is there's any method or workaround to make them rotate. Nevermind.
same here
it seems use a proxy
actually it works well when you use broker.serve inside the code, but the cli does not work well
@0xc0d @d-bo do you mean using broker.serve() inside code like this? https://github.com/constverum/ProxyBroker/blob/master/examples/proxy_server.py
I tried tracing the codes and found that it seems like rotating but, if you try to send requests one after one's response, you can see that it's reusing the previous one. Actually it's just keep getting one more new proxy every time it got a new client request when all pre-allocated proxies are in use, or reuse existing ones if there're any pre-allocated proxies available.
The demo looks like it's rotating is just because those requests are sending in parallel which caused an empty proxy queue. The proxy queue is always empty means it tries to fetch one more proxy every time.
Even you set limit = 100, but you always have only 1 concurrent connection, you'll be using the same proxy all the times.
If you only have max N concurrent connection, you'll have only N proxies rotating.
related lines of codes: https://github.com/constverum/ProxyBroker/blob/master/proxybroker/server.py#L37-L69 or https://github.com/constverum/ProxyBroker/blob/27f3766c9e8a48053669211893f5f681256d9f37/proxybroker/server.py#L37-L69
@bluet yes, You are right. I didn't try to fix the problem, because after all, I found out that public proxies are not quite well for my purpose. I tried rotating Tor server. It works. And yes, it's fast enough for me.