Large batch of notifications gets sent to subscriber in small parts rather than all at once

[jleveque]

When an application has subscribed for notifications and a large batch of notifications should theoretically be available, it appears as though the notifications "trickle in" to the application from the underlying Redis connection (~10 per second), rather than being received in a single, large batch.

This behavior was noticed while debugging an issue with caclmgd here: https://github.com/Azure/sonic-buildimage/issues/5275

[qiluo-msft]

I assume you are talking about caclmgrd. Could you try improve below code? https://github.com/Azure/sonic-buildimage/blob/bba5df05233e17ab3eb8cd378e4629923b30ff16/files/image_config/caclmgrd/caclmgrd#L553

After each select, there are lots of data in SubscribeStateTable. You can keep pop() until you get empty result.

[jleveque]

@qiluo-msft: @abdosi converted caclmgrd to use swsscommon. Could you also please provide an example of another application which "keeps pop() until you get empty result"?

[jleveque]

@abdosi: Have you confirmed that this is not actually an issue with swss-common, but rather with the way you were using it? If so, I will close this issue.

@jleveque as i mentioned here https://github.com/Azure/sonic-buildimage/pull/5621 with the pop until empty we are preventing update to iptable for x rules by x times.

In my experiment for the configuration where we just dump x rules all x notification are present in select object queue withy no delay and we are processing them continuously.

I feel my experiment proves there is no delay in swsscommon . Do you want any particular test to run ?

I am also planning to let Mellanox QA person to test with this change and let us know.