resque-clojure icon indicating copy to clipboard operation
resque-clojure copied to clipboard

Add support for agents watching specific queues

Open arohner opened this issue 13 years ago • 1 comments

In my environment, I have some jobs that can take 1 hr+, and some jobs that take 5 seconds. I also need to limit the number of concurrent 1 hr jobs, while allowing some of the fast jobs to continue running. I'd also like to avoid starvation, where a fast job is blocked by a slow job.

The obvious solution for me is to configure the agents independently. I'm thinking something like:

(start {["fast-queue1" "fast-queue2"] 5 ["slow-queue"] 3})

This would create 5 agents watching fast-queue 1 and 2, and 3 agents watching the slow queue. The one problem is this would break the :max-workers setting.

I'm willing to make all changes necessary for this, if you approve. (I already have all code done, except for the start configuration syntax).

arohner avatar Feb 16 '12 02:02 arohner

Hi, sorry for the slow response. I like the idea of avoiding starvation by using separate queues. I'd like to see what you have so far. Then we can work out how to handle configuration.

jxa avatar Feb 21 '12 22:02 jxa