ipyparallel icon indicating copy to clipboard operation
ipyparallel copied to clipboard

Retry engines that raise UnmetDependency

Open rueberger opened this issue 7 years ago • 0 comments

I'm using ipyparallel in a somewhat hackneyed scheme as a GPU cluster - upon job submission, each engine checks to see whether there are available GPU resources for running the job, and reports back to the scheduler if not. This can be accomplished with @depend, if the Scheduler can be set to not blacklist engines that raise dependency errors. What I'm really looking for is a resource_busy error though.

Is it possible to disable engine blacklist upon raising dependency errors? Or, even better, to more directly tell the scheduler that an engine is busy?

rueberger avatar Mar 21 '17 15:03 rueberger