bullmq icon indicating copy to clipboard operation
bullmq copied to clipboard

Is the concurrency configured in bull jobs global or local to a worker?

Open nadeesha opened this issue 3 years ago • 5 comments

It's not clear that this documented concurrency is applicable per worker, or per the queue globally. Any light on this is appreciated.

nadeesha avatar Apr 21 '22 08:04 nadeesha

The article is explaining the 2 ways you can achieve concurrency. Either with the concurrency factor per worker or with multiple workers. Not sure how this can be clarified, open for suggestions.

manast avatar Apr 21 '22 09:04 manast

@manast The concurrency factor is a worker option that determines how many jobs are allowed to be processed in parallel This is the main confusing part. In order to make it clear, at the very end of the sentence it would help to add across all of the workers or within a single worker.

kibertoad avatar Oct 02 '22 19:10 kibertoad

@kibertoad @manast a small follow up.

If I have workers pulling jobs from different queues. say workers w1, w2 are pulling jobs from q1, q2 respectively. here w1 and w2 are part of the same node process.

would w1 and w2 with concurrency 1, pull jobs simultaneously or not?

soumilbaldota avatar Mar 29 '25 04:03 soumilbaldota

hi @soumilbaldota in this case w1 and w2 are independent as they belong to different queues, so they will pull jobs simultaneously, 1 job each

roggervalf avatar Mar 29 '25 05:03 roggervalf

you can also read https://docs.bullmq.io/guide/queues/global-concurrency if you want to have a global concurrency in a queue

roggervalf avatar Mar 29 '25 05:03 roggervalf