connect icon indicating copy to clipboard operation
connect copied to clipboard

Add support for `NakWithDelay` for `nats_jetstream`

Open static-moonlight opened this issue 2 months ago • 3 comments

When NATS messages are naked, they are going to be re-delivered immediately (see here, as described for the Backoff parameter, or here, directly in the NATS source code).

Sometimes is makes sense to wait a little before processing a message again, for example when it involves talking to another system and the network is down. In such a case, chances are good that the network is still down, when retrying immediately.

It would be very helpful to have support for NATS' NakWithDelay function and a configurable delay for the nats_jetstream input processor. That would allow to design processors who behave like "I can't process this message right now, please task me again in 1 minute or something".

static-moonlight avatar Nov 12 '25 12:11 static-moonlight

Was just about to make a similar request. If no one gets to this before me I'll look into it and try to get a PR up next week for this

anthonyvitale avatar Nov 12 '25 21:11 anthonyvitale

Some minor digging leads me to think this is a bit more involved than I initially thought. We seem to still be on the legacy jetstream api which doesn't support nak with delay from what i can tell (https://github.com/nats-io/nats.go/blob/1949406fa68390b46b1963f0f4a04ec972c5152d/nats.go#L702C1-L719C20).

So we'll likely have to finish something like #3626 for this to be implemented.

anthonyvitale avatar Nov 12 '25 21:11 anthonyvitale

So we'll likely have to finish something like https://github.com/redpanda-data/connect/pull/3626 for this to be implemented.

It doesn't look like as if this is going to happen anytime soon ...

static-moonlight avatar Nov 13 '25 16:11 static-moonlight