unidist
unidist copied to clipboard
modin-project
Unified Distributed Execution
Reproducer: ```python import unidist import time @unidist.remote def sleep(seconds): time.sleep(seconds) @unidist.remote def f(arr): unidist.get(sleep.remote(0.2)) return sum(arr) def g(): obj_ref = unidist.put(100) arr = [obj_ref] for i in range(22): arr.append(f.remote(arr)) time.sleep(0.1)...
Blocking get to a worker fails inside a remote function, if the same worker sends back an operation to the worker that issues the blocking get. eg: If worker_A sends...
Pytest hangs after running all tests when running tests from a directory. This could be due to multiple instances of unidist.init() being called. Could be fixed by having a conftest...
While the option works for Ray backend, we should think of handling it for other backends.
Testing of actor for `multiprocessing` backend on Windows throwed the next errors: ```bash @pytest.mark.parametrize("is_use_options", [True, False]) def test_address_space(is_use_options): > actor0 = TestActor.options().remote(0) if is_use_options else TestActor.remote(0) unidist\test\test_actor.py:34: _ _ _...
Tests in `test_general.py` hang in case of using a small number of workers. https://github.com/modin-project/unidist/blob/bd45958838a62576b962ca9e19b39f7fb0e41161/unidist/test/test_general.py#L15-L24 Possible reason is in incorrect finishing worker-process in `multiprocessing` backend. To reproduce need to set `CpuCount...
We should add sections/pages on execution flows of unidist with different backends: * unidist_on_ray * unidist_on_dask * unidist_on_mpi * unidist_on_multiprocessing * unidist_on_python