cubed

cubed copied to clipboard

Use the `threads` executor to run some computations with free-threaded Python and see if it works. Ref: * [Free-threaded CPython is ready to experiment with!](https://labs.quansight.org/blog/free-threaded-python-rollout) * [py-free-threading.github.io](https://py-free-threading.github.io/) See also #497

tomwhite

Docs on how to write a new executor

1

There are a lot of parallel frameworks that Cubed Plans could be converted to (e.g. #499). We have executors for dask & beam, but instead of trying to write more...

TomNicholas

HPC executor

9

Some possibilities: * Job arrays: https://slurm.schedmd.com/job_array.html (pointed out by @jeromekelleher) * funcX: https://funcx.org/ * Parsl: https://parsl-project.org/ cc @TomNicholas

tomwhite

Ray Executor

6

In addition to accelerator support (e.g. via #304), Cubed could benefit ML users by providing [ray](https://github.com/ray-project/ray) executor: https://docs.ray.io/en/latest/ray-core/walkthrough.html Since Cubed is a serverless model, I bet it could get away...

alxmrs

Run tests on Python 3.13 (currently beta)

1

tomwhite

Use Zarr groups for outputs with multiple fields

1

We currently use a single Zarr array with a [structured data type](https://zarr-specs.readthedocs.io/en/latest/v2/v2.0.html#data-type-encoding) for storing intermediate outputs with multiple fields (such as the total and count when [computing the mean](https://github.com/cubed-dev/cubed/blob/e9de1aec8b1f86da20c128f4021a54351ccadcf6/cubed/array_api/statistical_functions.py#L34-L76)). Structured...

tomwhite

Implement NumPy's `__array_function__` protocol for array methods that are not in the Array API Standard

1

This is to support `nanmean`, `nansum` (see #153), and `pad` (#461). The previous attempt at this in #468 was reverted in #469, since it was causing problems when used with...

tomwhite

Predictive model to find optimal reduction parameters

4

This is a possible solution to #418 Our model aims to predict the optimal `split_every` value that makes the reduction as fast as possible. This parameter affects the input data...

balanz24

Re-implement `map_direct` functions using `general_blockwise`

1

The following functions are currently implemented using `map_direct`, which assumes that the input arrays have been materialized as Zarr arrays so they can be accessed directly, typically in a way...

tomwhite

This came up in the context of sgkit where we want to load Zarr arrays that have an object dtype (even if we don't do any processing on them).

tomwhite