[Bug]: Deduplication id not taking into account for Delayed task

[or-opus]

Version

v5.43.1

Platform

NodeJS

What happened?

We are using the debounce config for avoiding duplicated repetative tasks to be createad. seems like once a task is in delay a new task with the same debounce (deduplication id) is being created again.

How to reproduce.

create a task with deduplication id and put it in delay create a new task with the same deduplication id task is created algoht there is an exiting task with the same deduplication id in delay

my task options { "attempts": 3, "removeOnFail": { "count": 1000000 }, "deduplication": { "id": "e8a14d47-783e-444c-8536-64a2e5f4e32e" }, "delay": 36000000, "jobId": "142416-retry", "removeOnComplete": { "count": 100 }, "backoff": { "delay": 600000, "type": "exponential" } }

Relevant log output

Code of Conduct

• [x] I agree to follow this project's Code of Conduct

[manast]

Can you please provide a complete test case that demonstrates the issue?

[BarakOpus]

Hey @manast , I think I found the bug. The case that @or-opus mention is when a job with deduplication id failed, and after a new delayed job created with the same deduplication id. When I manually deleted the failed job, the lua script that remove the job from redis call the removeDeduplicationKey function and it delete the deduplication key from redis but the deduplication key value (job-id) is actually for the other (delayed) job.

The lua function: https://github.com/taskforcesh/bullmq/blob/master/src/commands/includes/removeDeduplicationKey.lua

I think the solution can to check before removing the deduplication key to check that the deduplication-key value is match to the removed job-id.

[BarakOpus]

Suggested code fix:

local function removeDeduplicationKey(prefixKey, jobKey)
  local deduplicationId = rcall("HGET", jobKey, "deid")
  if deduplicationId then
    local deduplicationKey = prefixKey .. "de:" .. deduplicationId
    local currentValue = rcall("GET", deduplicationKey)
    if currentValue == jobKey then
      rcall("DEL", deduplicationKey)
    end
  end
end

[roggervalf]

since v5.50 this should be fixed