Bottleneck before re-queueing (Sidekiq::Client.push) because of @redis.lock?

[katpadi]

When we tried to scale up, the Sidekiq workers were not maximized and it seemed like there's a bottleneck in the grouping process. Is this expected?

My configuration:

grouping:
  :poll_interval: 1      # Amount of time between polling batches
  :max_batch_size: 100   # Maximum batch size allowed
  :lock_ttl: 1           # Batch queue flush lock timeout job enqueues

[giovannelli]

Hi @katpadi, we are experiencing the same issue, the grouping queue seems to be slow, I added a lot more Sidekiq processes listening on empty queues to leave room to the grouping poller, but with no luck.