Causes duplicate jobs to be created on restart

[adambutler]

trafficstars

We were using version 1.0.0.alpha.1 with Sidekiq 5.5.9 when we noticed a strange issue where we would get many copies of jobs in our retry queue.

We upgraded sidekiq to 7.1.2 but the issue persisted and could be reliably replicated until we removed sidekiq-throttled.

Steps to replicate:

1. Queue a job that will raise in our case this job was not using sidekiq-throttled
2. Allow it to go into the retry queue
3. Restart the sidekiq process
4. Many copies of the job get created

Whilst investigating we removed this dependency and the issue stopped presenting itself.

[ixti]

Can you please test with >= v.1.0.0? If you can provide a snippet that I can use to replicate that will help significantly. So far I've tried this one:

require "bundler/setup"

require "sidekiq"
require "sidekiq/throttled"

Sidekiq::Throttled.setup!

class SampleJob
  include Sidekiq::Job
  include Sidekiq::Throttled::Job

  sidekiq_throttle concurrency: { limit: 3 }

  def perform
    raise "oops"
  end
end

SampleJob.perform_async if ENV["ENQUEUE"]

Sidekiq is 7.2 and sidekiq-throttled tried both alpha and release versions:

1. run with ENQUEUE=1 bundle exec sidekiq -r ./demo.rb
2. kill the proceess with Ctrl+C
3. run bundle exec sidekiq -r ./demo.rb
4. retry set contains only one job