laravel-queue-rabbitmq
laravel-queue-rabbitmq copied to clipboard
vyuldashev
added support for blocking channel->wait
Using blocking call can speed up the latency for the queue processing significantly.
@arep During development of Consumer I tried with setting nonblocking to false and not all features of Laravel worked.
That might be, but it would be nice to have the option to use it anyway. For me at least it is important that the messages are processed as quickly as possible, and then I must use blocking call. Maybe mention in the docs that blocking call might not work for all features. Do you remember what features that did not work?
Hi @arep, is this still something you would like added to the library?
Yes. We are using this feature in production and it has worked great.
Well, i've been trying to push same solution (https://github.com/vyuldashev/laravel-queue-rabbitmq/pull/296) in late of 2019 :)
@arep do you have some links / docs explaining the perf value of doing this?
Can you detail in which cases this brings such value?
Are you using prefetch-count? Without knowing much more about your case, it seems like loosing a loop or 2 because of non-blocking calls but then getting 300 messages to work on would end up being not much of an issue.
I'm all for improving performance, before jumping in, I'd like to have a clear case for when this option makes things better.
@arep do you have some links / docs explaining the perf value of doing this? Can you detail in which cases this brings such value? Are you using
prefetch-count? Without knowing much more about your case, it seems like loosing a loop or 2 because of non-blocking calls but then getting 300 messages to work on would end up being not much of an issue. I'm all for improving performance, before jumping in, I'd like to have a clear case for when this option makes things better.
We are using rabbitmq to do RPC calls from the webserver to a service that runs database querys. The reason is that some queries can take a long time, 5-60 sec, while most is completed within milliseconds. On one of our setups there are around 150 queries pr/sec. The query service instances are waiting for messages on the queue with blocking call so they process them as fast as possible. If we did not use blocking call, there would be a wait loop and any queries that use sub-second time would be slowed down significantly. Any non-blocking loop would have to have some time of sleep or else the servers CPU would go 100% all the time. Only one query at a time can be processed by one query service instance, so prefetch can not be higher than 1. Since we can't know in advance if the query is slow or fast.
Hope that clarifies the use case.
Understood, so it's a queue with low traffic but where the latency of processing the job matters.
I'll look into this a bit more.