classroom

classroom copied to clipboard

Feature request :sparkles:

Currently classroom uses sidekiq queues inefficiently. This is our current queue setup: https://github.com/education/classroom/blob/4f9db7e412c57ce8aeb328a355b223e0a7254546/config/sidekiq.yml#L7-L14

This is what the Sidekiq docs say about having lots of queues:

I don't recommend having more than a handful of queues. Lots of queues makes for a more complex system and Sidekiq Pro cannot reliably handle multiple queues without polling. https://github.com/mperham/sidekiq/wiki/Advanced-Options#queues

One way we could improve this would be to use the regular 3 queues (critical, default, and low).

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

This issue has been automatically marked as stale because it has not had recent activity. Thank you for your contributions.

This issue has been automatically marked as stale because it has not had recent activity. Thank you for your contributions.

Hi! I can help with this ✋

I think I can do this one. Have you done any work on this @v4vasanth?

---

@BenEmdon So, your idea here is to just create these three queues that you've mentioned and re-assign the workers? I thought about doing something like this:

:queues: # https://github.com/mperham/sidekiq/wiki/Advanced-Options#queues
  - [critical, 3]
  - [default, 2]
  - [low, 1]

And assigning the Workers that have a queue with priority 3 to the critical queue and so on.

Other option would be this setup:

:queues: # https://github.com/mperham/sidekiq/wiki/Advanced-Options#queues
  - critical
  - default
  - low

But from the Sidekiq documentation we have that:

This means that any job in the default queue will be processed only when the critical queue is empty.

I don't like the idea of only processing from default when critical is totally empty. So I will go for the option one.