unidist
unidist copied to clipboard
PERF-#239: Improve task scheduler for MPI backend
What do these changes do?
Improve task scheduler for MPI backend, Replaced the round robin algorithm to submit tasks, with a getting a worker rank with maximum data share. Data is pushed to workers in the put method. Data id and data sizes would be stored in object store.
- [ ] passes
flake8 .
- [ ] passes
black .
- [ ] signed commit with
git commit -s
- [ ] Resolves #239
- [ ] tests added and passing
- [ ] module layout described at
docs/developer/architecture.rst
is up-to-date