Broker: Allow message scheduled enqueue time to be expressed with TimeSpan

[SeanFeldman]

Related to the issue https://github.com/Azure/azure-service-bus-dotnet/issues/160 that might require broker side changes (this issue).

[nemakam]

@SeanFeldman , do you expect the variance in machine clocks (for UTC) be considerably more than the variance in the latency of the send scheduled message request? Could you provide more details on when you encountered this issue.

[SeanFeldman]

This happens already when you have scheduled en-queue time very close to the current. I've already provided a description in the other client repo, here.

It boils down to not relying on the local clock when asking to enqueue a message in the future where a server clock is used. TimeSpan is better for those scenarios when the clock skew is significant.

[SeanFeldman]

@nemakam, here's another request for TimeSpan based API.

[NickAskew]

@nemakam, I have the same issue as @SeanFeldman, setting [EnqueuedTimeUtc] involves having to anticipate how far out the clock on the machine posting the BrokeredMessage is from the ServiceBus clock, meaning I have to stick on the safe side when wishing to delay messages by specifying minutes, rather than a few seconds for redelivery. I understand that redelivery of BrokeredMessages may itself take a few seconds after the specified [EnqueueTimeUtc] but if a relative enqueue time was supported then at least it can be guaranteed that the message will not be redelivered in less time than was intended given clock skew.

I have attached some timings of my local clock vs ServiceBus clock, I sent 25 messages each scheduled by an incrementing number of seconds which reveals clock out of sync by >16 seconds in less than 2 days since the clock was synced with time.windows.com ServiceBus ScheduledEnqueueTimeUtc out of sync by 16 seconds over 2 days.txt

[NickAskew]

Real world issue: When sending SMS via SMPP I asyncronously send a SubmitSm PDU and expect a SubmitSmResp to be returned on completion of the call. The SubmitSmResp contains a unique id which I persist to my DB. Asyncronously the server also sends me a DeliverSm PDU with an intermediate receipt for the SubmitSm PDU I just sent, this receipt contains the unique id I get from the SubmitSmResp, so I can update the DB with the receipt.. however given all these methods are being run asynchronously it can happen that I get a DeliverSm PDU returned BEFORE my call to send a SubmitSm has finished, meaning I need to stick the DeliverSm in a SB Queue for redelivery until such time as I have persisted the unique id from the Submit Sm. All these message delivery statuses are exposed via a GUI to a user (who has just sent the SMS) and expects almost real time feedback on the delivery status of the SMS to the handset it was sent to.. which of course in this event will be a couple of minutes because I have to requeue the message in minutes rather than seconds. Reposting the message after 5 seconds would be perfect in this situation. Yes I could sleep for XXX and block the code receiving the statuses, but not ideal. Yes I could hold onto the DeliverSm locally - at the risk of losing it if the service goes down before anything can get persisted in the db (the whole point of using the SB in the first place)

[SeanFeldman]

Any updates on this issue?

[SeanFeldman]

@axisc would be nice to get this one and remove the dependency on DateTime.

[EldertGrootenboer]

We have opened an investigation task for this in our backlog, and will update this issue when we have more information.

[EldertGrootenboer]

Thank you for your feedback on this item, it helps us in our efforts to continuously prioritize the different requests we get from all our various feedback channels. We have added this feature in our backlog, however we currently don't have an ETA on when development might start on this. For now, to help us give this the right priority, it would be helpful to see others vote and support this feature, as well as explain their scenarios.

[EldertGrootenboer]

This feature in our backlog, however we currently don't have an ETA on when development might start on this. For now, to help us give this the right priority, it would be helpful to see others vote and support this feature, as well as explain their scenarios.

[NickAskewGH]

I assume you just don't understand the reason for failure?

1. I put a message onto the bus, for delivery at DATETIME
2. The bus receives my request, puts the request into its queue along with the DATETIME I specified, but now this DATETIME is relative to the BUS's current date+time, not my date+time (assuming our clocks are not in sync)
3. The bus see's the DATETIME has arrived (based on the bus's current date+time, not my date+time, because our clocks are not in sync) and puts the message back onto the queue to be delivered.
4. I receive the message immediately, rather than a 10 second wait (in the case where clock skew between my machine and the BUS's current date+time are more than 10 seconds difference)

Now if my request to re-deliver the message was for in the future +10 seconds, and the bus's current date+time is more than 10 seconds different from my clock, then it is likely that the message will be placed immediately back onto the queue for me to pick up, without the 10 second wait. This is the problem.

Why do we want a timespan?

1. I put a message onto the BUS, with a +10 second timespan.
2. The bus accepts my +10 seconds, looks at its own current date+time, adds the 10 seconds to that and sticks it in its queue.
3. The bus's scheduler see's the message is now ready to deliver and sticks it back onto the queue for me to pick up.
4. I pick the message up no sooner than 10 seconds from when I put it on the queue, regardless of the difference in both our clocks.