sidekiq-throttled icon indicating copy to clipboard operation
sidekiq-throttled copied to clipboard

question: is it expected that throttled jobs run out of order?

Open josh-m-sharpe opened this issue 1 year ago • 1 comments

It seems that if I continue to refresh /sidekiq/queues/high - I see a lot of change. And I took a look at the codez and it looks like the gem might work by:

  1. letting sidekiq pluck jobs off the queue normally
  2. re-enqueue them if the job should be throttled

...which would explain why my queue seems so volatile.

Is it possible to avoid this rotation and work the jobs in the order they were enqueued?

Fwiw, I'm throttling multiple jobs with:

Sidekiq::Throttled::Registry.add(:my_throttle,
  :concurrency => {
    :limit => 1
  },
  :threshold => {
    :limit => 5,
    :period => 1.minute
  }
)
 sidekiq_throttle_as :my_throttle

josh-m-sharpe avatar Jul 04 '22 18:07 josh-m-sharpe

Hi. You're right, current version rotates the queue. Throttling happens on per-worker class, not queue. There's no way to overcome this right now. But I have this on my list of things to improve – a way to configure throttling on a per-queue basis.

ixti avatar Jul 04 '22 19:07 ixti