vectordotdev
AMQP source: Configure prefetch count
A note for the community
- Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
- If you are interested in working on this issue or have submitted a pull request, please leave a comment
Use Cases
While reading messages from AMQP source using Vector, we observed that we can't configure the prefetch_count attribute which helps in limiting the number of unacknowledged messages being sent to the consumer. (https://www.rabbitmq.com/docs/consumer-prefetch, https://www.rabbitmq.com/docs/confirms#channel-qos-prefetch-throughput) The prefetch_count is by default configured as 0 which means infinite number of messages could be sent to the consumer without receiving acknowledgement. This could crash the consumer.
This is my understanding. Please let me know if I am missing something or if not, then I think this attribute should be added for the AMQP source.
Attempted Solutions
No response
Proposal
No response
References
No response
Version
0.40.0
Thanks for opening this @amanmahajan26 !
I agree we should expose this configuration. I see the library we are using supports it: https://docs.rs/lapin/latest/lapin/protocol/basic/struct.Qos.html#. Happy to see a PR if you or anyone else feels motivated.
As a workaround it looks like you can configure a default on the broker side: https://docs.rs/lapin/latest/lapin/protocol/basic/struct.Qos.html#
