cubed
cubed copied to clipboard
cubed-dev
Bounded-memory serverless distributed N-dimensional array processing
Hey @tomwhite, I'm having trouble getting the Array API tests to run locally. I'm trying to debug this failure with my M1-mac PR: https://github.com/alxmrs/cubed/actions/runs/10971436728/job/30466556319?pr=2 I'm not sure how to debug...
For a while I've been thinking that Cubed and `dask.array` are potentially on a path of convergent evolution to a very similar design. After talking to @phofl at length at...
Currently, projected memory checks are performed as the array API operations are called: ```python import cubed spec = cubed.Spec(work_dir="tmp", allowed_mem=100) # not enough memory! a = xp.asarray([[1, 2, 3], [4,...
The current text on [why cubed?](https://cubed-dev.github.io/cubed/getting-started/why-cubed.html) is written at a very high level. It's very interesting for people who think about different distributed systems, but it's not super clear to...
In discussions with Tom and Tom, it was unclear where this example involving the use of the `"dragon"` start method should land. This PR is intended to help further those...
In #554 and #555 @applio started to add support for running Cubed on Dragon, with the intention of allowing users to run Cubed on HPC. One thing that's super nice...
@dcherian's PR to xarray https://github.com/pydata/xarray/pull/9320 adds a new primitive to the ChunkManager, which presumably should be added to cubed too?
Fixes #310 This is useful for xarray testing. I had hoped to be able to implement this by writing an executor that raises when its `execute_dag` method is called -...
From a [recent run](https://github.com/cubed-dev/cubed/actions/runs/10308706981/job/28536745015?pr=548): ``` ______________________ test_stragglers[timing_map0-10-2] _______________________ tmp_path = PosixPath('/tmp/pytest-of-runner/pytest-0/test_stragglers_timing_map0_100') timing_map = {0: [10]}, n_tasks = 10, retries = 2 @pytest.mark.parametrize( "timing_map, n_tasks, retries", [ ({0: [10]}, 10, 2),...
This updates the slow tests (that run test_mem_utilization.py) to run on both zarr-python v2 and v3.