ProxyBroker icon indicating copy to clipboard operation
ProxyBroker copied to clipboard

How to add timeout to main loop

Open Jealrock opened this issue 7 years ago • 3 comments

Im followed the examples and faced the problem that if i got many unique proxies, queue stay empty for a long time until exception is rised, how can i reduce this time with a custom timeout?

Jealrock avatar May 08 '18 22:05 Jealrock

Read docs please: http://proxybroker.readthedocs.io/en/latest/api.html

Case you're using CLI commands, just add --timeout=10 or other, in seconds. In case you're using the python class just add timeout=10 to the arguments.

erm3nda avatar May 08 '18 23:05 erm3nda

This is timeout for requests, but i need timeout for asyncio.Queue, or a way to add timeout without using it

In the basic example this line awaits too long if i have enough processed proxies

proxy = await proxies.get()

I tried using asyncio.wait instead of asyncio.gather but it had no effect

Jealrock avatar May 09 '18 09:05 Jealrock

Sure, always better if you tell most of details of your question to avoid mistakes and wasting time with unwanted replies :-D

From https://docs.python.org/3/library/asyncio-queue.html

asyncio queue API was designed to be close to classes of the queue module (Queue, PriorityQueue, LifoQueue), but it has no timeout parameter. The asyncio.wait_for() function can be used to cancel a task after a timeout.

As far i can see this is the only chance. Asyncio has no the common Queue.put() timeout params.

erm3nda avatar May 09 '18 21:05 erm3nda