rain
rain copied to clipboard
Task pinning to workers
Add task attribute to allow only a subset of workers. An empty attribute
Usage:
- Working on files local to a worker (either reading or writing).
- Useful for
openandstoretasks (but not limited to them).
- Useful for
- Selecting suitable worker subset for a taks when (virtual) resources are not appropriate.
- However, for GPUs and other features, we should use resources.
- Testing.
Constant data objects already have a similar feature for placement, but this API is not planned to be published (and it is not really relevant for computed data objects).
Alternative: Every worker could be a resource, required by the task. However, this is not ergonomic and we do not have a clear picture of how to do (virtual) resources in the right way.
@spirali What is the status of this?
This is still implemented only for scheduler testing