OptimalBits
Cannot dynamically increase concurrency of a queue
Description
Suppose a queue is declared as queue.process(concurrency, foo) with some concurrency.
If there are additionally more jobs to process later on, and I have to increase the concurrency of my queue, I need to restart my application and re-initialise my queue with the new concurrency.
-
I could not find anything in the doc as to how to do this dynamically, so is there any API or any way that can dynamically increase the concurrency of the queue without having to restart the application?
-
If I redefine the
processwithqueue.process(new_concurrency, foo), are there any repercussions to the jobs that are currently running in this queue? Or is this the way I can increase the concurrency dynamically? -
Any API to get the current concurrency of a queue? WIP or future plans or against Bulls ideology?
Bull version
bull: "^3.3.7"
I would second this. My use-case is IO-bound, so much simpler if there was a way to update the concurrency vs spinning up new processes.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
![stale[bot] avatar](https://avatars.githubusercontent.com/in/1724?v=4 "Published by a pub.dev verified publisher"){kind=small}
I will leave this one as an enhancement, it would be a new method to change the concurrency setting of a queue.
related https://github.com/OptimalBits/bull/issues/290 much needed improvement.
Why not just start the queue with the highest concurrency setting you would like? I do not see the point in having less concurrency than the max you want to have. Maybe there is something I am missing.
I will leave this one as an enhancement, it would be a new method to change the concurrency setting of a queue.
Has this been implemented?
It is implemented in BullMQ https://api.docs.bullmq.io/classes/Worker.html#concurrency No plans ATM to backport it.