Concurrency management

[kibertoad]

Readme mentions "Queue storage policies to support a variety of rate limiting, debouncing, and concurrency use cases", but I couldn't find any details on supported concurrency management options in the documentation.

Is it already possible to say "I want to spin up 5 workers per service node to be consuming events from queue X"? (concurrency = 5)

Is it already possible to say "I want to spin up 5 workers per service node, up to 2 workers per project, to be consuming events from queue X"? (concurrency = 5, groupConcurrency = 2)

If not, is this something that you would be open to being implemented?

[timgit]

Readme mentions "Queue storage policies to support a variety of rate limiting, debouncing, and concurrency use cases", but I couldn't find any details on supported concurrency management options in the documentation.

This is my fault for not being especially dilligent on updating the readme. A concurrency feature I had for years was removed in v10.

Is it already possible to say "I want to spin up 5 workers per service node to be consuming events from queue X"? (concurrency = 5)

I now recommend using multiple workers for concurrency. I didn't include this API pg-boss, but in the app I was using it with, I added an abstraction that basically did a for loop over a concurrency property that called work() to add however many workers are desired. For example, work('my-queue', { concurrency: 5 }) would be a convenience wrapper over calling work() 5 times.

Is it already possible to say "I want to spin up 5 workers per service node, up to 2 workers per project, to be consuming events from queue X"? (concurrency = 5, groupConcurrency = 2)

How are you defining project and groupConcurrency?

If not, is this something that you would be open to being implemented?

Yes 👍

[kibertoad]

I now recommend using multiple workers for concurrency. I didn't include this API pg-boss, but in the app I was using it with, I added an abstraction that basically did a for loop over a concurrency property that called work() to add however many workers are desired. For example, work('my-queue', { concurrency: 5 }) would be a convenience wrapper over calling work() 5 times.

Any reasons not to have it in pg-boss itself?

How are you defining project and groupConcurrency?

Probably the BullMQ way where whoever is doing the scheduling is aware how to resolve the group is going to be the easiest:

const job1 = await queue.add(
  'test',
  { foo: 'bar1' },
  {
    group: {
      id: 1,
    },
  },
);

const job2 = await queue.add(
  'test',
  { foo: 'bar2' },
  {
    group: {
      id: 2,
    },
  },
);

Implementing something on manager side (so that it know which group id resolver to use for which queue) is likely to be fiddlier, and worker likely is getting involved in this workflow too late for resolver to be placed with it.

And just to clarify the intention - idea behind group concurrency is to prevent any possibly big tenants from clogging the pipe for everyone - can be a large customer/project/whatever, hence group is whatever organizational unit that developer would like to apply concurrency limits by.

[timgit]

Any reasons not to have it in pg-boss itself?

This seems reasonable to add. It's been plenty of time since the last concurrency feature was deprecated.

And just to clarify the intention - idea behind group concurrency is to prevent any possibly big tenants from clogging the pipe for everyone - can be a large customer/project/whatever, hence group is whatever organizational unit that developer would like to apply concurrency limits by.

This is similar to #446. This is also something worth adding some disclaimers about. Whenever "round robin" and fairness algorithms are discussed, it's always one-sided towards being concerned about the "noisy neighbor" problem where a large tenant is hogging all the bandwidth. And I think it's right to have a solution ready to counter that problem. What the disclaimer needs to point out is that sometimes the larger tenant is paying for more infra, so they rightfully deserve more compute and a higher priority.

A good example is a 1 paying customer competing for queue processing with 1000 free trial accounts. This example shows that the goal is not always 100% equal processing across tenants, but rather a balance between the "large tenant" problem vs. the "large number of tenants" problem.

[kibertoad]

I agree, caveats need to be documented. Do you think we need to support an even more sophisticated tiered concurrency model? e. g. you not just round-robin across the groups, but you have credit allocation per group, and you can specify which tier this job is, meaning that bigger customer can potentially get bigger concurrency budget allocation.

[timgit]

I wouldn't want to block round robin because of something more complicated, no.

It's good enough to match other package and queue offerings, but I at least want to document tradeoffs at a minimum so it's clear what the user is signing up for.

Also, I'm not saying the PR won't be accepted without docs. Part of this exercise is being able to bounce ideas off of someone else.