Tom White

That's great Deepak! We definitely want a multi-level merge chunks function. I think that Cubed's `reduction` function is almost what we want - rather than reducing along an axis we...

See https://github.com/cubed-dev/cubed/tree/tree-merge-chunks. The trick is to use a `combine_size` of -1 along an axis to mean the size shouldn't change.

Amazing! You certainly picked one of the most difficult functions to implement for your first contribution @dcherian. It's the first use of recursion to build a Cubed graph 😄

Thanks @alxmrs for opening this issue. I'm not familiar enough with JAX or GPUs to answer these questions, but I'd be happy to support or discuss an initiative in this...

Thanks for the pointers @alxmrs. Thinking about how Cubed might hook into this, the main idea in Cubed is that every array is backed by Zarr, and an operation maps...

> Where is a good place to make this kind of call within Cubed? Is this something that should be handled by an Executor (this seems not so ideal)? Possibly...

Nice!

I think compiling the (Cubed optimized) blockwise functions using AOT compilation (as you mentioned in https://github.com/cubed-dev/cubed/issues/490#issuecomment-2247599475), and then exporting them so they can run in other processes (https://jax.readthedocs.io/en/latest/export/export.html) may be...

Yes, I saw this and think it could be very interesting for Cubed. > Presumably any improvement to the I/O speed would considerably increase the performance of cubed? Hopefully, but...

I tried swapping out Zarr for TensorStore here: https://github.com/tomwhite/cubed/tree/tensorstore. All unit test pass, except for `test_resume` (since TensorStore doesn't have a way of accessing Zarr's `nchunks_initialized`). There were a few...