QueueClassic
Allow for transient processing of jobs
It would be convenient to have a single entry point into QC that processes all jobs, then exits (i.e. a non-daemon based worker). Something resembling:
$ QC_EXIT_ON_COMPLETION=true rake qc:work
or perhaps a new rake task all together?
The use case here is to reduce the need to have a long-lived daemon to keep alive for systems with low numbers of jobs, or jobs that can wait for a cron-based process to spawn and work on them. On Heroku this would allow users to have an hourly task with Scheduler to process the queue instead of a whole worker dyno.
As I alluded in a tweet, you can almost do this today.
worker = QC::Worker.new
QC.count.times {woerker.work}
The problem is that QC::Worker#work spins on the worker's @running instance variable in the case that there are no jobs. So if a job is worked between the time your call QC.count and worker.work then you will end up blocking the thread.
I think it would be nice if QC::Worker#work did not block and instead returned nil in the case it was unable to find a job. That would make the aforementioned snippet safe.
Thoughts?
That solves the problem, though it requires a small bit of knowledge about the internals of QC. I guess it's nothing to wrap your code snippet in a rake task or some other helper method, though.
Great point. If we had the right primitives, we could easily wrap them up into a QC provided Rake task.
+1 from me.
Perhaps we need some kind of a lock mechanism to prevent other workers from doing work while the queue is emptied? (the case that @ryandotsmith described). This would also help with multi-threaded environments.
:+1: would be nice to have this.
@rwdaigle are you interested in working on a patch?
@senny I'll put it on my list, but suspect it will not be in the near term that I'll be able to work on it.
@rwdaigle I took a stab at this https://github.com/QueueClassic/queue_classic/pull/256 let me know if this addresses your needs.