nats-server icon indicating copy to clipboard operation
nats-server copied to clipboard

Extend a way: For pending messages that have not responded to ack, force them to time out immediately instead of waiting for the set time ack timout

Open GuangchaoDeng opened this issue 4 years ago • 3 comments

Feature Request

I expect to use NATS jetstream as the underlying transfer for file synchronization or database data synchronization, perhaps the file is large and the database records collected are large; It's not very friendly to process at the moment It may be useful to support transaction mechanisms. ACK timeout is more difficult to control ACK timeout is too long or too short is not suitable

Use Case:

Proposed Change:

Extend a way: For pending messages that have not responded to ack, force them to time out immediately instead of waiting for the set time ack timout

Who Benefits From The Change(s)?

In this way, I can set the ack timeout long enough, so that the immediate reply ack is similar to commit, and the immediate timeout is similar to rollback

Alternative Approaches

GuangchaoDeng avatar Apr 09 '21 03:04 GuangchaoDeng

For what you are trying to do, there is a better approach by using ephemeral consumers with no acks and re-creating the consumer if you miss a message (chunk).

Will try to put together an example.

derekcollison avatar Apr 09 '21 13:04 derekcollison

@derekcollison this was my suggestion too - though for various reasons he seem to want to have a WQ for the message store, presumably since we cant say purge all messages matching some criteria such a subject or something.

ripienaar avatar Apr 09 '21 14:04 ripienaar

In case pull-based consumer, the filer subject is a regex subject, and the messages in each subject are in ordered and should be processed in order. In case the consumer is stopped and restarted when processing messages, we recreate the consumer and it should be re-received the pending ack messages immediately instead of received the next messages in wrong order. I think we might have 2 ways but i do not know is it possible or not:

  1. Is possible to set max pending ack message for a subject instead of regex subject, so we can set max pending ack message to 1. Currently this setting only effects on the consumer, not on a single subject.
  2. Whenever reconnect pull-based consumer, all the pending ack massages should be re-delivered immediately.

Thanks!

datphan310 avatar Jul 16 '22 04:07 datphan310