Why not ? add delay option when sending messages?

[apuatcfbd]

Proposed change

Talking about JetStream specifically (because I'm using it). When messages are received by a consumer, that can nac the message with delay. so the delay (some sort of support) is in the server. so why not allow delaying in the first place when messages are sent? I want to propose adding an option to add delay to messages. The delay will work for the delivery. If a message gets added with the delay of 1 minute then the message should be delivered >= 1 minute.

I'd also like to extend the delay option in a flexible way. That can be with

1. time.Duration so when sending that'll look like send the message in 30 minutes
2. time.Time that'll be send the message on 2024-10-20 12:35:40

In my app (where I'm using JetStream I also need a scheduler with relatively safe scheduling. Like: I need to delete some specific records from DB every day at 12 AM. For this, I need to write my own scheduler or use a premade one from Awesome Go. But the issue is I need Redis or a DB to store those schedules (so those don't get lost when the app crashes/restarts). I've looked into the package Asynq that thing has all I need (message with the above kind of delays & a scheduler) but notice that need Redis, so I can't use NATS KV as I've to use Redis anyway. Now either I've to leave beloved NATS or all other lib that'll allow me to do something later & scheduled way.

So what's the problem? You see a lot of similar things getting added to the project. I need some functionalities of NATS but I can't fulfill my needs. In a single projects view JetStream can be replaced with Asynq, Redis needs (except Asynq usage) can be replaced with NATS KV... you get the picture.

I think you'll agree with me that there's no question about how important is to have a scheduler and a queue with delay support in a real-world app. NATS (server) is already a very powerful tool. So what I'm proposing is

1. Allow the delay in the first place when sending messages
2. Make the delay flexible by allowing delay with time.Duration & time.Time
3. Add a scheduler or way to make the message continuously deliver the message at a predefined schedule.

I may be asking too much or this could go against the goal of NATS, In that case, I'm sorry, but in my opinion, these options are very much needed in many applications & almost all of this can be done by extending current NATS server (assumption) as that thing runs in a different server.

Use case

Message with delay:

1. With time.Duration This is needed in many cases where something needs to be done after some time another thing is done. Like Sending an email to somebody 5 minutes later after something happened in the server.

2. With time.Time This is needed when something needs to be done later but in a specific date-time. That could be 3 months 2 weeks n hours n minutes later. Instead of calculating the time.Duration of it, it's very convenient to just give it the time.Time

Scheduler: This is needed to carry out repetitive tasks like preparing report, clearing tables/ logs, etc. in a predefined schedule. Generally, this is wanted for an infinite number of times. The scheduler can work with time.Duration and cron, the following is a complex schedule job.

cron (0 22 * * 1-5) This cron resolves to At 22:00 on every day-of-week from Monday through Friday. This can't be expressed in time.Time or `time.Duration. So this is very useful.

Contribution

Yes, as much as I can.

[derekcollison]

We are looking into allowing embargo'd messages.

[OneSeven]

@derekcollison There is an urgent need for a function to deliver delayed messages (delay queues). Currently, when using NATS, the delayed message depends on the msg.NakWithDelay function, which means that a message must be consumed meaninglessly at least once. It would be great if the msg.NakWithDelay function could be extracted and the consumption delay could be set directly!

[apuatcfbd]

@derekcollison This was need for one of my project as well, I managed to do it with following (workaround)

1. Send the message with a metadata that specifies when to deliver the msg (the delay)
2. Receive the msg and then NakWithDelay & use the delay from the metadata

[derekcollison]

As I said, we are looking into embargo'd messages and possibly more. /cc @ripienaar

[ripienaar]

Some (very) rough ideas here https://github.com/nats-io/nats-server/issues/6884

Would value feedback to hear if that would help your case.

[gatspy]

It's been how many years, and we're still discussing this issue. Giving up using NATS, this will be the biggest root cause.

Now my system uses nats as the messaging engine, but I also wrote a mechanism based on Redis delayed message processing, damn it.

[jnmoyne]

The ability to delay message publishing was added to JetStream in nats-server version 2.12:

https://github.com/nats-io/nats-architecture-and-design/blob/main/adr/ADR-51.md

https://github.com/nats-io/nats-server/releases/tag/v2.12.0