nack icon indicating copy to clipboard operation
nack copied to clipboard

Lack of subject update Changing nats filter subjects

Open dpalat opened this issue 2 years ago • 15 comments

Problem description

Hello team, I am having a issue trying to update the subject of a consumer with nack, so the issue is related with the change of one subject to a multiple subjects filter and also when I want to add another simple subject filter into the array of subjects.

The only way to apply the changes is deleting the consumer and then restarting nack so it is reacreating the consumer with the correct subject.

Any idea or plan to fix this issue? do you have a better workaround than delete the consumer and restart the nack pod?

issue 1: image

issue 2: image

Nack logs: E1102 10:34:19.371115 1 controller.go:460] failed to process consumer: failed to update consumer "workload" on stream "MAIN": consumer subject filters cannot overlap (10138)

What version amI using?

nack version: natsio/jetstream-controller:0.13.0 nats version: 2.10.3 yml version: jetstream.nats.io/v1beta2

Is this defect reproducible?

I think yes

Given the capability you are leveraging, describe your expectation?

a kick fix and final fix in the next version of nack

dpalat avatar Nov 02 '23 11:11 dpalat

The steps I've used (these are even worse, so your steps would be much better though still not great!):

  • Delete the consumer in NATS itself
  • kubectl delete consumers.jetstream.nats.io <consumername>
  • kubectl apply

lukaso avatar Nov 09 '23 12:11 lukaso

@lukaso thanks for the reply, the deletion of the consumer works fine but very difficult to do that when you consumer is running in productions with a lot of traffic, instead of that as a Workaround you can edit the subject consumer using the command with the Nats cli anyways also there is a bug trying to use the Nats cli with the Args, but you can do the workaround using consumer edit with json configuration, look the this issue in the Cli repo for more details https://github.com/nats-io/natscli/issues/918

The steps I've used (these are even worse, so your steps would be much better though still not great!):

  • Delete the consumer in NATS itself
  • kubectl delete consumers.jetstream.nats.io <consumername>
  • kubectl apply

@lukaso thanks for the reply, the deletion of the consumer works fine but very difficult to do that when you consumer is running in productions with a lot of traffic, instead of that as a Workaround you can edit the subject consumer using the command with the Nats cli anyways also there is a bug trying to use the Nats cli with the Args, but you can do using consumer edit with json configuration, look the this issue in the Cli repo for more details https://github.com/nats-io/natscli/issues/918

dpalat avatar Nov 09 '23 15:11 dpalat

Hey. Thanks for reporting the issue. I'm looking into it.

Jarema avatar Nov 10 '23 08:11 Jarema

I found a bug in jsm and issued a fix.

https://github.com/nats-io/jsm.go/pull/501

Jarema avatar Nov 10 '23 10:11 Jarema

Thanks a lots @Jarema 🎉🎉🎉

dpalat avatar Nov 15 '23 09:11 dpalat

Hello, any more updates on this? I believe jsm needs a new release and then nack needs to reference that new version. Is there anything more?

DiogoRoloOS avatar Dec 15 '23 15:12 DiogoRoloOS

No. It's just that. Do we have a schedule for releasing jsm @ripienaar ?

Jarema avatar Dec 17 '23 21:12 Jarema

Not a schedule as such but there will be a release before mid jan

ripienaar avatar Dec 18 '23 06:12 ripienaar

Not a schedule as such but there will be a release before mid jan

Is a fix comming? We just ran into the issue...

mgross-ebner avatar Jan 12 '24 14:01 mgross-ebner

I just ran into this as well. Anything I can do to help?

mjhoffman65 avatar Feb 01 '24 20:02 mjhoffman65

Looks like a jsm release hasn't been cut yet.

image

mjhoffman65 avatar Feb 01 '24 20:02 mjhoffman65

+1 on running into this issue. We also will be deleting and recreating the consumer, but as others have mentioned it is a less than ideal process in production environments.

CTJeffries avatar Feb 01 '24 20:02 CTJeffries

Here's a gist I'm using to work around this problem. Remember this is code that works for me but no warranty (since it is mutating your system). https://gist.github.com/lukaso/347c8d9afdc6b21ce8fbd15456594480

lukaso avatar Feb 02 '24 11:02 lukaso

Can we get the latest ESM release out?

drabelogarner avatar Feb 16 '24 17:02 drabelogarner

Created this PR to fix this issue: https://github.com/nats-io/nack/pull/170

DiogoRoloOS avatar Mar 14 '24 10:03 DiogoRoloOS