cubed
cubed copied to clipboard
cubed-dev
Bounded-memory serverless distributed N-dimensional array processing
One of the things that became apparent from AGU was that we need more examples! Here are a few that I'd like to add: * [x] Zarr. Read a regular...
I've noticed this line causes an error on my machine: https://github.com/cubed-dev/cubed/blob/1ac50612e48882554f08951e0072559ead6b2e71/cubed/tests/test_core.py#L592 Looking around, I think this is a problem in Python 3.11, see https://github.com/python/cpython/issues/100316. My developer environment specs ```shell (cubed)...
We should add the [Array API type hints](https://github.com/data-apis/array-api/tree/main/src/array_api_stubs) to the Cubed implementations. (Suggested by @TomNicholas.)
Following #650 In particular, need to explain how it's implemented as a series of general blockwise copy operations.
This issue tracks the work needed to support the remaining [linear algebra functions](https://data-apis.org/array-api/latest/extensions/linear_algebra_functions.html) from the [Python array API standard](https://data-apis.org/array-api/latest/index.html). The current Cubed coverage status for the linalg extension can be...
### Concept Icechunk solves the problem of handling incremental updates to Zarr stores, meaning that users can transparently track changes to datasets at the chunk level. Often real-world data pipelines...
One of Matt Rocklin's blog posts [mentions](https://docs.coiled.io/blog/2024-eoy.html#whats-next) Prefect's "serverless work pool" https://docs.prefect.io/v3/deploy/infrastructure-examples/serverless My (hazy) understanding of what this is is a way to deploy a cluster in the cloud on-demand...
This is a proof-of-concept for #467. It takes advantage of the fact that Globus Compute has [an implementation of Python's `concurrent.futures.Executor`](https://globus-compute.readthedocs.io/en/latest/executor.html). I tested it by running a Globus Compute endpoint...
**Note that the Python array API standard v2024 is still in [draft](https://data-apis.org/array-api/draft/index.html).** The following is based on [closed issues](https://github.com/data-apis/array-api/milestone/4?closed=1), and will be updated once the 2024 standard is released. ##...
Hi, I ran into an error when trying to save the xarray to zarr. It appears that if i load the xarray dataset using dask chunk, this problem somehow does...
