rrqueue icon indicating copy to clipboard operation
rrqueue copied to clipboard

dependent tasks

Open richfitz opened this issue 8 years ago • 1 comments

I'd like to support dependent tasks so that we can write:

enqueue(expr, depends=c(task_id1, task_id2))

To support this, a dependent task would be "queued" immediately, but not actually pushed onto the queue (generated by rrqueue_key_queue) though we would have to hash which queue it is intended to go onto.

Then as a job completes, we could deal with it.

The trick would be making sure that the lookup didn't get too costly. We could store "is blocking" against running jobs or "waiting on" against the new jobs. The latter is probably easier to get right, the former could be done more efficiently perhaps.

richfitz avatar Sep 28 '15 16:09 richfitz