arq icon indicating copy to clipboard operation
arq copied to clipboard

work distribution among workers

Open sushmakoulgi opened this issue 6 months ago • 1 comments

I have 500 jobs in my queue , worker as non-blocking function to wait for 5 sec. (asyncio.sleep(5)) , when I spawn 3 workers to do the jobs, I see one workers takes most of the work, and 2 other sits idle most of the time. How to make sure all 3 workers picks up job concurrently?

here are my worker configurations job_timeout = 100 keep_result = 0 poll_delay = 0.5 max_jobs = 50

Image

sushmakoulgi avatar Jun 14 '25 21:06 sushmakoulgi

Hello @sushmakoulgi

If you share your code, that would help a lot :)

dnikolayev avatar Sep 24 '25 15:09 dnikolayev