Tom White

Add page on optimization to user guide

3

See #381. This relies on #396 to make it easier to cross-refer operation IDs in the debug output to the visualizations.

Improve Lithops logging configuration

3

Lithops [configures Python logging](https://github.com/lithops-cloud/lithops/blob/55392b0f875651568542e339985866903555d550/lithops/utils.py#L176-L227) itself, which can be handy, but equally can make it hard to configure logging for other (non-Lithops) modules. For example, using `logging.basicConfig` in your code doesn't...

Tune straggler mitigation algorithm

1

The algorithm for [launching backups](https://github.com/tomwhite/cubed/blob/929f52cad0642d7adb11fe6c2ff07e9b8741387f/cubed/runtime/backup.py) has not been tested very much. We should try it on a few runtimes and tune the parameters so it works reasonably well without user...

From @TomNicholas's code to generate the chart at https://xarray.dev/blog/cubed-xarray#cubed-results

Update the documentation to reflect the changes due to the work in #339. This should include end user docs on how to configure/control optimization, as well as more developer-focused design...

Profile Cubed task memory usage with Memray

1

I've been using [Memray](https://bloomberg.github.io/memray/) to profile the memory usage of Cubed tasks running locally using a local Lithops executor (since it runs tasks in a separate process). This give great...

Query optimization

3

There are a couple of possibilities for doing query optimization that have come up recently. [Dask-expr](https://github.com/dask-contrib/dask-expr) will support arrays soon (https://github.com/dask-contrib/dask-expr/issues/446). It would be interesting to see if the expression...

Low-latency storage backends

21

So far we've only used cloud storage (S3 and GCS) for storing intermediate Zarr data in Cubed. It would be interesting to try other storage backends that have lower latency....

Move main array namespace

4

Currently all the array functions are in `cubed.array_api`. This was created to follow the naming pattern for the new array API in `numpy.array_api`. Since then, however, there has been a...

This would 1. allow the use of different storage backends in addition to the regular Zarr library - in particular #187. 2. convert the backend array API being used (NumPy,...