resque-clojure
resque-clojure copied to clipboard
Add support for agents watching specific queues
In my environment, I have some jobs that can take 1 hr+, and some jobs that take 5 seconds. I also need to limit the number of concurrent 1 hr jobs, while allowing some of the fast jobs to continue running. I'd also like to avoid starvation, where a fast job is blocked by a slow job.
The obvious solution for me is to configure the agents independently. I'm thinking something like:
(start {["fast-queue1" "fast-queue2"] 5 ["slow-queue"] 3})
This would create 5 agents watching fast-queue 1 and 2, and 3 agents watching the slow queue. The one problem is this would break the :max-workers setting.
I'm willing to make all changes necessary for this, if you approve. (I already have all code done, except for the start configuration syntax).
Hi, sorry for the slow response. I like the idea of avoiding starvation by using separate queues. I'd like to see what you have so far. Then we can work out how to handle configuration.