cubed icon indicating copy to clipboard operation
cubed copied to clipboard

Delayed-like functionality

Open tomwhite opened this issue 2 years ago • 1 comments

There are some cases where it would be useful to have a way of setting up a parallel map over inputs, then calling compute on it later - like Dask Delayed.

In particular this would be needed to support xarray's open_mfdataset (see https://github.com/pydata/xarray/pull/7811), or creating Kerchunk files in parallel.

The main question is what this looks like and where it belongs.

tomwhite avatar Sep 20 '23 13:09 tomwhite

Here's a basic proof of concept: https://github.com/tomwhite/cubed/commit/5f6e38e8e790a05298ca9b7ee89a55e7b7e9edfd

tomwhite avatar Sep 20 '23 13:09 tomwhite