Feature request: Update Azure Service Bus to support duplicate detection when using Spring Boot Starter for Azure Service Bus JMS

[heatzync]

Description

This is more a feature request than a bug report.

Our project has been implemented based on this article: https://docs.microsoft.com/en-us/azure/developer/java/spring-framework/configure-spring-boot-starter-java-app-with-azure-service-bus

Messages are sent and received and is working as expected.

We however have the additional requirement now to detect and discard duplicate messages which I believe is supported by Azure ServiceBus. Based on some articles on stack overflow, we have tried setting various message IDs using a org.springframework.jms.core.MessagePostProcessor, e.g.:

jmsTemplate.convertAndSend(theQueue, someEvent, message -> {
    String duplicatePreventingId = generateDuplicatePreventingId(someEvent);

    message.setJMSMessageID(duplicatePreventingId);
    message.setJMSCorrelationID(duplicatePreventingId);
    message.setStringProperty("MessageId", duplicatePreventingId);
    return message;
});

Actual Behavior

1. The three message IDs that we set when sending are: JMSMessageID, JMSCorrelationID and MessageId
2. The org.apache.qpid.jms.JmsSession class in qpid-jms-client overrides the JMSMessageID as part of the sending process (after the post processor was called)

Object messageId = null;
if (!disableMsgId) {
    messageId = producer.getMessageIDBuilder().createMessageID(producer.getProducerId().toString(), messageSequence);
}

...

// Set the message ID
outbound.getFacade().setProviderMessageIdObject(messageId);

3. The outbound.getFacade().setProviderMessageIdObject(messageId) statement above results in the _messageId field being populated on the org.apache.qpid.proton.amqp.messaging.Properties class in proton-j
4. On the receiving/consuming end the message is received, but the JMSMessageID is now a random generated string whilst the JMSCorrelationID and MessageId properties are left intact.
5. Due to the JMSMessageID being overridden to a randomly and unique generated ID I suspect the duplicate detection is not working on the Azure ServiceBus broker.

Expected Behavior

Would it be possible for Azure ServiceBus to not rely on theJMSMessageID or MessageId (as it is referred to in the Azure SDK), but instead expect an alternative (additional) property ID to be populated, say for example DuplicatePreventingId? Then the above code will look like this:

jmsTemplate.convertAndSend(theQueue, someEvent, message -> {
    String duplicatePreventingId = generateDuplicatePreventingId(someEvent);
    message.setStringProperty("DuplicatePreventingId", duplicatePreventingId);
    return message;
});

and duplicate detection should work when used with the azure-servicebus-jms-spring-boot-starter.

Why can't we rely on setting the JMSMessageID?

According to the JavaDoc of the Message#setJMSMessageID method in the javaee API:

This method is for use by JMS providers only to set this field when a message is sent. This message cannot be used by clients to configure the message ID. This method is public to allow a JMS provider to set this field when sending a message whose implementation is not its own.

Some even regard it as illegal:

What you are trying to do is illegal regarding the JMS specs,"JMSMessageID" can not be changed/set.

[heatzync]

It's been a month since I opened this feature request and I haven't received any response. Did I at least post it on the correct project?

[joshfree]

Hi @heatzync, thank you for reaching out to us via this GitHub issue -- and thank you @EldertGrootenboer for transferring the issue to the correct repo. @yiliuTo could you please follow up with @heatzync?

[yiliuTo]

Hi @heatzync , sorry for the late response. And for your feature request

Would it be possible for Azure ServiceBus to not rely on theJMSMessageID or MessageId (as it is referred to in the Azure SDK), but instead expect an alternative (additional) property ID to be populated, say for example DuplicatePreventingId?

This should be a feature request for the Azure Service Bus serive side, just as what you mentioned, to set the JMS message ID is not recommanded and even regarded as illegal in JMS specs. Thus what I can do is to contact to the service team to ask for your feature request. Therefore, could you help to share your email address then I can add you into the mail list for further communication?

[heatzync]

Hi @yiliuTo. Thank you for your response.

How do I send you a DM with my e-mail address? Seems like GitHub has removed that feature...

[yiliuTo]

Hi @heatzync , then would you like to email our team with your request? Our team email is [email protected]

[heatzync]

Thanks @yiliuTo. I've sent an e-mail to [email protected]

[yiliuTo]

Hi @heatzync now that we have contacted the ASB service team for this feature request, thus is this issue ok to be closed?

[heatzync]

Hi @yiliuTo. That's an option provided I have some way of keeping track of the progress of the feature request logged with the ASB service team. How can I keep track of the progress?

[yiliuTo]

Hi @heatzync , in the mail the service team has mentioned that they will create a tracking item, you may ask them whether that item can be public and shared to you. And if not, how about asking them whether there is some other way to track that request? And of course you can keep this GitHub issue open if there is no better alternative to track.

[heatzync]

Thanks @yiliuTo. @EldertGrootenboer has agreed to update this GitHub issue once they have done their investigation. I will therefore keep it open at least until the feedback has been added.

[EldertGrootenboer]

Thank you for your feedback on this item so far. We are currently actively investigating the possibilities around this feature, however we currently don't have an ETA on when development will start on this. We encourage everyone to share the scenarios where they would like to use this feature, to help us shape it in the best way.