Shawn Yang

This is pretty tricky, the `index` in `DataFrameIndexAlign` reduce chunk is a tuple for following graph:  The tuple index for chunk index break the assumption in ray shuffle,...

Can we add other scheduler like `Slurm` here using execution API? @fyrestone

> https://github.com/wjsi/mars/tree/feature/shuffle_svc Is there a design doc for this?

Another issue is Ray client server bottleneck:  In the Ray Task mode, a large number of ObjectRefs are held in the supervisor. If the supervisor is created in the...

 Seems mars hang at ray call.

 core owker log: ``` (mars-py3.8-dev) chaokunyangdeMacBook-Pro:mars chaokunyang$ grep -nr "580d375e7573d16eb11bba3846f2bc9f44b924b40100000001000000" /tmp/ray/session_latest/logs/ /tmp/ray/session_latest/logs//python-core-driver-01000000ffffffffffffffffffffffffffffffffffffffffffffffff_12183.log:3016:[2022-05-17 21:55:33,621 D 12183 6422828] reference_count.cc:181: Adding owned object 580d375e7573d16eb11bba3846f2bc9f44b924b40100000001000000 /tmp/ray/session_latest/logs//python-core-driver-01000000ffffffffffffffffffffffffffffffffffffffffffffffff_12183.log:3123:[2022-05-17 21:58:15,824 D 12183 6422227] reference_count.cc:1365: Published message...

Seems >  core owker log: > > ``` > (mars-py3.8-dev) chaokunyangdeMacBook-Pro:mars chaokunyang$ grep -nr "580d375e7573d16eb11bba3846f2bc9f44b924b40100000001000000" /tmp/ray/session_latest/logs/ > /tmp/ray/session_latest/logs//python-core-driver-01000000ffffffffffffffffffffffffffffffffffffffffffffffff_12183.log:3016:[2022-05-17 21:55:33,621 D 12183 6422828] reference_count.cc:181: Adding owned object 580d375e7573d16eb11bba3846f2bc9f44b924b40100000001000000 > /tmp/ray/session_latest/logs//python-core-driver-01000000ffffffffffffffffffffffffffffffffffffffffffffffff_12183.log:3123:[2022-05-17...

> @Catch-Bull Will ray support `batch.put`? If not, I'll close this issue

@jyizheng Which mars version are you using? I tested your script with mars master(df1492c4974bb027e7916ae116b12a340e3abdea) and it passed without error. My test setup is macos 2018 and python 3.8, and ray...

I reproduced the error using `pymars==0.8.1` and `ray==1.9.2` on may mac, will ping you after I located the issue @jyizheng