cloudamqp
`max_unacked_messages` queue parameter
Is your feature request related to a problem? Please describe.
We're trying to implement a semaphore-like pattern to limit the maximum number of messages being processed (by virtue of being unacked) at any given time (to control resource consumption of other services downstream).
Describe the solution you'd like
We'd like to have a max_unacked_messages queue parameter. This is the limit of unacked messages in a queue, across all consumers/channels. If the number of unacked messages has reached the limit, no more messages will be dequeued until one is acked again.
Describe alternatives you've considered
Alternatives we've looked into:
- Implementing outside of LavinMQ: this is not very efficient because it would involve dequeueing a message, then checking whether it's allowed to process it (e.g. increment a distributed counter) and if not rescheduling it again.
- Using a queue length limit: unacked messages are not counted towards the length limit, so this won't work.
- Using channel or consumer prefetch: this doesn't limit the total number of prefetched/unacked messages across all channel to the queue.
Hello, thanks for the report! Best would be to solve this on your clients with a pre-fetch value so you cannot consume more messages without acknowledging them, but I understand if this does not work in your case.
To be candid, adding a queue parameter is not prioritised currently and we would need to look closer at an implementation strategy we would want to commit to. Just top of mind this should be a pretty simple addition (famous last words 😅), but not without question. One concern we discussed is adding another parameter check in an already pretty busy deliver loop.
