significance of messageWaitDuration in MessageHandlerOptions

[backendjavadeveloper]

i am trying to register my message Hander for azure servicebus and i see the following description about one of the constructors of messageHandlerOptions class:- maxConcurrentCalls maximum number of concurrent calls to the onMessage handler autoComplete true if the pump should automatically complete message after onMessageHandler action is completed. false otherwise. maxAutoRenewDuration - Maximum duration within which the client keeps renewing the message lock if the processing of the message is not completed by the handler. messageWaitDuration duration to wait for receiving the message

Assuming i have a pile up of messages in my queue and i switch on my listener code, this is what i understand will happen :

1. maxConcurrentCalls is basically the number of messages that will be picked or basically the number of threads that are gonna get spawned to make sure to read the messages from the queue.
2. autoComplete : what do we mean by pump here?? this is basically the ack that we send either auto or manual. but what do we mean by pump in this messaging context?
3. maxAutoRenewDuration: this is the time period within which the consumer has to send ACK to servicebus else the lock will be released and other consumers will get the message. rt??
4. messageWaitDuration: what is this for??? i was playing around with this and it looks like the longpoll timeout value?? is this correct?? what are the implications if i keep this value to a high value like Duration.ofDays(10) and what if i keep this very low i.e. Duration.ofSeonds(10)??

Also, As i understand for the standard tier, the pricing is based on the number of operations performed. So if i keep the Duration a long value lets say some days so does that mean that the number of operations are gonna be less when compared to a lesser value of duration???

I might be completely off the grid here so if you can please help me understand the significance of messageWaitDuration param, and what are the outcomes if i keep this value in days vs keeping this value short like seconds or minutes. Also as i need to take the pricing into account as well, can you also clarify that this param does or doesnt add up in the number of operations in backgroud thus resulting in cost??

Thanks

[nemakam]

Thanks for this issue. I take this as a documentation improvement issue.

maxConcurrentCalls is basically the number of messages that will be picked or basically the number of threads that are gonna get spawned to make sure to read the messages from the queue.

One you have registered your message handler, you will start receiving messages infinitely until you call close(). But based on application capabilities, you may want to limit the number of messages that are handed out to you at a time, i.e., flow control. So yes, as you mentioned, if maxConcurrentCalls=5, that means 5 threads would be spawned and each of them would do a receive() and hand over the message to you. Only when the message has been processed, will a thread try to receive the second message. So essentially its the max number of times your method is called at any point in time.

autoComplete : what do we mean by pump here?? this is basically the ack that we send either auto or manual. but what do we mean by pump in this messaging context?

When you register a message handler, we internally start threads to receive messages from service and hand it over to your handler. This is called as pump. It pumps the messages to your handler. And autoComplete=false means your handler is expected to complete() the message. autoComplete=true means the "pump" will automatically complete the message for you if you don't throw an exception from your handler.

maxAutoRenewDuration: this is the time period within which the consumer has to send ACK to servicebus else the lock will be released and other consumers will get the message. rt??

Correct.

messageWaitDuration: what is this for??? i was playing around with this and it looks like the longpoll timeout value?? is this correct?? what are the implications if i keep this value to a high value like Duration.ofDays(10) and what if i keep this very low i.e. Duration.ofSeonds(10)??

Looks like this was a param that need not be exposed at all. Might deprecate this parameter completely. In the case of high throughput scenario, it doesn't matter whether you wait for 10 seconds or 10 days. In the case of low throughput scenario, we keep track of whether we have already sent a request for a message or not. If we have, then the next receive() internally will realise that we are waiting for a message already, and hence we don't send a new request. Hence it becomes a client side operation only. So calling it every 10 seconds doesn't matter. It wont increase the number of operations on the service as it doesn't even reach the service. Every request that reaches the service is counted as an operation and will be billed. But passing a value of 10 days will end up causing trouble. Since after 10 minutes of inactivity, the service will close the link, the message will not be received. And the next request to service would be sent only after 10 days, which is not ideal :)

[nemakam]

Action items:

1. Simplify documentation words
2. Deprecate messageWaitDuration

[backendjavadeveloper]

Hi @nemakam , Thanks for replying back. So i got a question on the scheduled messaging feature that Azure Service Bus provides. This is in regards to point where we talk about messageWaitDuration param. This is how i use Azure Service Bus for my application -

1. I use azure servicebus only for the scheduled messaging feature.
2. So i will only send requests to azure servicebus when my app server receives a request.
3. As per your statement on the messageWaitDuration param, this is what i can understand. a) the java client internally calls the receive calls to get the messages. So this parameter tells the time to wait (before killing the link between app and servicebus) till the message is received from the servicebus queue. If i set this as 1 minute, then it means that java client will send the short poll requests every 1 min.

Is that correct??

Also one quick question on similar lines, what happens if there is no message sent / received from azure servicebus for some days. Will my connection terminate and not reconnect again automatically?? coz i do understand that an idle timeout value would be configured on your LoadBalancers and thus treat my connection as a dead connection. But the java client will take care of reconnecting the same rt??

[nemakam]

@backendjavadeveloper

Will my connection terminate and not reconnect again automatically??

Yes. Connection and link gets terminated as we have an idle timeout. The link is closed by service bus after 10 idle minutes. But when you retry a receive() command, the link should be automatically reestablished. You need not do this yourself. However, looks like there has been an ongoing issue with the reconnection, but that's just a bug which will be fixed.

If i set this as 1 minute, then it means that java client will send the short poll requests every 1 min.

As of now, no. The client internally does a long poll and doesn't send a new request every minute. But this might change.

[mikeppp]

So this is confusing..

What we are doing is:

a. Basically just waiting for messages to come in on an the async receive

What we are finding is that we are getting following error when there are no messages flowing:

com.microsoft.azure.servicebus.primitives.ServiceBusException: Error{condition=amqp:connection:forced, description='The connection was inactive for more than the allowed 60000 milliseconds and is closed by container 'LinkTracker'. TrackingId:26e6feed77614de0a260d3c3d2115026_G6, SystemTracker:gateway7, Timestamp:6/29/2018 7:11:24 PM', info=null}

Doesn't the defeat the ENTIRE purpose of registering a listener??? How do I make the listener be able to wait indefinitely on a queue for incoming messages???

also are getting:

WARN com.microsoft.azure.servicebus.primitives.CoreMessageReceiver - No messages received from 'sl_inbound'. Pending receive request timed out. Returning null to the client.

our use case:

_queueClient.registerMessageHandler(new IMessageHandler() { @Override public CompletableFuture<Void> onMessageAsync(IMessage iMessage) { .. do stuff }

Thank you!

[mikeppp]

So with more testing, here is the strange thing..

It behaves as required, i.e even after all disconnect error messages it does receive again.

Why all the error messages?? It's really filling up the log even though its working.

Thank you

[nemakam]

We just emit log4j logs. Just disable the logs unless you need them for debugging... Rely on the exceptions thrown by the client and not the logs for general purposes.

[backendjavadeveloper]

Hi @nemakam , i had raised one more ticket https://github.com/Azure/azure-service-bus-java/issues/242 and here i am getting a reply as the requests made internally coz of param messageWaitDuration are actually chargable!!! Can you please clarify on this and confirm which is the correct behaviour???

[sanppatil]

Hi @nemakam,

I'm reading through your one of above comments and felt like we (Working at one of airline) are also facing similar issue in our production case.

Yes. Connection and link gets terminated as we have an idle timeout. The link is closed by service bus after 10 idle minutes. But when you retry a receive() command, the link should be automatically reestablished. You need not do this yourself. However, looks like there has been an ongoing issue with the reconnection, but that's just a bug which will be fixed.

We deployed Topic.SubscriptionClient on "App Service - Web Jobs" environment in continuous mode. We found that all of sudden SubscriptionClient stops receiving messages from Topic's subscription. We are using Sessions and its possible that during night hours, incoming message traffic is less or none and listener for given session can go into idle time for couple of hours.

We suspect that this idle time is causing to break connection to Service Bus. And after some time, irrespective of thousands of messages present in topic, It don't pick up anything or we dont observe anything like reconnection happening. As temporary work around, we are restarting SubscriptionClient (i.e. webjob) after seeing such issue. It resolves problem and again sporadically re-appears (After some days/hours) and we start seeing this problem of client not picking any messages.

At presently, We are using maven dependency for 1.2.5. Whether upgrade to 1.2.7 will resolve this problem? Can you please elaborate ongoing issue you wanted to highlight in your comment?

Appreciate your help. Thanks in advance.

Thanks

[nemakam]

Yes. Upgrade to latest client should help. Take a look at #236

[sanppatil]

Thnx you @nemakam, I will try this and monitor results for week to confirm back.

[krangasaamy]

Hi @nemakam,

https://github.com/Azure/azure-service-bus-java/issues/219

I have referred the above link and the person saying even after upgrading to 1.2.7, still facing the issues... can you please throw some comments on that...