Deduplication/Debounce features

[blackmouse572]

Problem

Since we are using event-based flows, debounce/deduplication can help us to control how the flow handled and as the bullmq featured:

is s a process where job execution is delayed and deduplicated based on specific identifiers. It ensures that within a specified period, or until a specific job is completed or failed, no new jobs with the same identifier will be added to the queue. Instead, these attempts will trigger a deduplicated event.

Proposed Solution

The bullmq solved this already, we can simulated it with emit functions

• Add deduplication options to emit

await emit({
        topic: 'ts.feeding.reminder.enqueued',
        data: {
          petId: pet.id,
          enqueuedAt: Date.now()
       },
      deduplication: { id: pet.id, ttl: 5000, extend: false, replace: true}
})

For more details, you can read here

---

Since we already have a local state implementation, we can integrate it internally to store the id of event and check it before going to next event received

Alternatives Considered

No response

Additional Context

No response

Willing to Help Implement?

• [ ] I would like to work/help with this

[rohitg00]

Nice suggestion, let us discuss this internally and get back to you. Thanks!

[sergiofilhowz]

We're going to implement deduplication and messageGroupId (for FIFO queues)

[ytallo]

hey @blackmouse572 it's a great suggestion. As @sergiofilhowz said, we are implementing deduplication and FIFO queues support, so we will implement your suggestion.