mars

mars copied to clipboard

# Background Large-scale distributed computing systems may fail due to various reasons, including network problems, machine failures, and process restarts. Network failures can cause nodes and workers to fail to...

chaokunyang

# Background When building chunk graph, Mars chunks won't be added to graph if some chunks not used by downstream, because mars build graph back to first. This will make...

chaokunyang

[BUG] test_ownership_when_scale_in hang

3

**Describe the bug** When running test case `DEBUG_OSCAR=1 pytest -v -s mars/deploy/oscar/tests/test_ray.py::test_ownership_when_scale_in`, it hangs occasionally. **To Reproduce** To help us reproducing this bug, please provide information below: 1. Your Python...

chaokunyang

[storage] Add batch.put API for storage

4

**Is your feature request related to a problem? Please describe.** `Ray.put` with owner will issue an sync rpc to owner, when there are many obejcts to put, `Ray.put` will become...

chaokunyang

Currently `RayExecutionContext.get_chunk_meta` is not supported, which will make any operands relied on this API failed on tiling, such as when call `DataFrame.groupby`: ``` df = md.DataFrame(mt.random.rand(300, 4, chunk_size=100), columns=list("abcd")) df["a"],...

chaokunyang

[BUG] to_csv raises FileNotFoundError: [Errno 2] No such file or directory

1

**Describe the bug** The problem is that when execute to_csv in a distributed cluster, it may raises FileNotFoundError: [Errno 2] No such file or directory.  It is because the...

fyrestone

Currently df.groupby() in Mars only implements ``as_index``. ``sort`` is added but not implemented yet, while ``level`` is not implemented which is also useful. What's more, when ``GroupBy`` object is generated,...

wjsi

Information to Collect =============== * Execution details of every job * Data read amount (with confidence) * Execution cost (with confidence) * Network transfer amount (with confidence) * Peak memory...

wjsi

Currently, Mars worker allocates CPUs for tasks via ``DispatchActor``, who allocates CPUs in an exclusive manner, that is, when a CPU is allocated to a task, it is removed from...

wjsi

**Is your feature request related to a problem? Please describe.** Some operands, such as summation over a long series of chunks, can be started and run partially when some data...

wjsi