cubed
cubed copied to clipboard
cubed-dev
Bounded-memory serverless distributed N-dimensional array processing
@yuvipanda pointed out that the name "cubed" is almost impossible to Google - there are so many other projects called cubed. (In our field "cubed" would also likely get confused...
I've been inspired by @thodson-usgs's Virtualizarr / Cubed rechunking example work and figured it's time to give Cubed a shot. Wondering if it's possible to add some documentation on how...
It would be nice to be able to do the following (like Dask): ```python with ProgressBar(): res = a.compute() ``` (Related to #538)
Fixes #538 Adds a page of docs covering `visualize` and progress bars.
Working my way through understanding cubed / cubed-xarray. I'm trying to get an example working of modifying the chunking of an Xarray dataset and writing it to Zarr. When I...
# Rechunk a virtual dataset This example demonstrates how to rechunk a collection of necdf files on s3 into a single zarr store. First, lithops and VirtualiZarr construct a virtual...
This allows users to use jit or aot compilation during the dag finalization process within executors. It should work straightforwardly on jax/numba style jit compilation. It's possible, but maybe ugly,...
I'm beginning to explore #304 in greater depth. Since the only local GPU I have access to is an M1 chip (I have an M1 Macbook Air), I thought I...
One thing that's really obvious at Scipy this year is that people absolutely love [DuckDB](https://duckdb.org/). The reason is that although DuckDB doesn't offer multi-machine parallelism so can never scale horizontally,...
