substrate does not allow non durable subscriptions to Nats streaming

[slawo]

Connection to Nats streaming are always done using durable groups. This is problematic in cases where the client ID of the subscriber is not the same on reconnection (new pod name for example), or when we don't want to have more than one client for a given durable subscription.

It is preferable to connect to non group subscriptions when a unique instance is consuming events from a queue (ex most projectors in the current configuration)

https://github.com/uw-labs/substrate/blob/67626b38e3a2382235b2307e2b82d50f297cec6b/natsstreaming/nats_streaming.go#L248

[mjgarton]

Can you elaborate on the problem please? What actually goes wrong in your application(s) that you need to fix? This issue is phrased like a solution, but I don't understand the problem.

[slawo]

In case of our projectors we don't want to have more than one consumer for a given sequence of events. The group subscription (conn.QueueSubscrib) allows multiple consumers for the same stream of events. This creates the potential for missed messages on one consumer while they are are consumed by another one. It also creates the potential of stale messages resulting if late redeliveries (broken order) in case of a consumer crashing/and reconnecting with a new client ID.

[mjgarton]

Let me understand first. Are you starting multiple consuming applications (or multiple instances of the same application) with the same QueueGroup ?