mars
mars copied to clipboard
Add backend for xarray
Is your feature request related to a problem? Please describe.
We can implement a xarray backend in Mars, refer to https://xarray.pydata.org/en/stable/internals/how-to-add-new-backend.html .
https://xarray.pydata.org/en/stable/internals/how-to-add-new-backend.html
This doc has 2 parts, the first said how to implement a new backend to read a new dataset; the 2 part is about how to implement a BackendArray that is so called duck array.
Mars tensor is already a duck array that acts just like numpy ndarray, so it can already work well with xarray, e.g. xr.DataArray(mars_tensor)
can work well.
However, xarray currently does not expose any parallel backend and just integrate tightly with dask in this part.
My plan is creating a submodule mars.contrib.xarray
to replace xarray itself, and the parallel method like chunk
, map_blocks
etc will be implemented on Mars.