ducttape icon indicating copy to clipboard operation
ducttape copied to clipboard

Throttling a single task

Open jhclark opened this issue 12 years ago • 0 comments

It might be useful to allow a maximum number of instances of a single resource-hogging task to run at a single time.

Example: Task 1 involves a large amount of disk IO (e.g. building phrase tables) and Task 2 involves a large amount of network overhead. Running multiple instances of either task in parallel will be slower than running serially. However, having an instance of task 1 in parallel with task 2 is fine.

It would be nice to define a mechanism that allows a scheduler (possibly a ducttape-internal scheduler) to enforce this constraint.

jhclark avatar Jun 19 '12 15:06 jhclark