mars icon indicating copy to clipboard operation
mars copied to clipboard

Add backend for xarray

Open qinxuye opened this issue 2 years ago • 1 comments

Is your feature request related to a problem? Please describe.

We can implement a xarray backend in Mars, refer to https://xarray.pydata.org/en/stable/internals/how-to-add-new-backend.html .

qinxuye avatar Mar 31 '22 06:03 qinxuye

https://xarray.pydata.org/en/stable/internals/how-to-add-new-backend.html

This doc has 2 parts, the first said how to implement a new backend to read a new dataset; the 2 part is about how to implement a BackendArray that is so called duck array.

Mars tensor is already a duck array that acts just like numpy ndarray, so it can already work well with xarray, e.g. xr.DataArray(mars_tensor) can work well.

However, xarray currently does not expose any parallel backend and just integrate tightly with dask in this part.

My plan is creating a submodule mars.contrib.xarray to replace xarray itself, and the parallel method like chunk, map_blocks etc will be implemented on Mars.

qinxuye avatar Apr 07 '22 08:04 qinxuye