exq icon indicating copy to clipboard operation
exq copied to clipboard

How to avoid duplicate jobs?

Open jaimeiniesta opened this issue 6 years ago • 2 comments

What is the proper way to check if a given job is already enqueued with the same arguments?

jaimeiniesta avatar Jun 04 '19 08:06 jaimeiniesta

I don't know if there is a proper way but I'd do something like:

{:ok, jobs} = Exq.Api.jobs(Exq.Api, "your-queue")
job_you_care_about = Enum.find(jobs, fn j ->
  j.class == "Elixir.Module.Name" && j.args == [arg1, arg2]
end)

There are a whole bunch of interesting functions on Exq.Api that could be useful:

iex(3)> Exq.Api.
Server                     busy/1
clear_failed/1             clear_processes/1
clear_retries/1            clear_scheduled/1
failed/1                   failed_size/1
find_failed/2              find_job/3
find_retry/2               find_scheduled/2
jobs/1                     jobs/2
processes/1                queue_size/1
queue_size/2               queues/1
realtime_stats/1           remove_failed/2
remove_job/3               remove_queue/2
remove_retry/2             remove_scheduled/2
retries/1                  retry_job/2
retry_size/1               scheduled/1
scheduled_size/1           scheduled_with_scores/1
start_link/0               start_link/1
stats/2                    stats/3

coop avatar Jun 28 '19 06:06 coop

The other thing to keep in mind is that exq seems to guarantee "at least once" job execution so you should be writing your jobs for idempotence.

coop avatar Jun 28 '19 06:06 coop