cubed icon indicating copy to clipboard operation
cubed copied to clipboard

Cubed with coiled executor example clarification

Open norlandrhagen opened this issue 4 months ago • 1 comments

Hey there @tomwhite,

I was interesting in trying cubed for rechunking using the coiled executor. This works fine, but ~~I don't see any coiled functions created in the coiled dashboard. Wondering if there is some additional config I need?~~

Edit: Adding from_array_kwargs={"spec": spec} seemed to fix it!

MRE

# cubed_version: '0.22.0'
# coiled_version: '1.120.0'
# xarray_version: '2025.9.0'


import xarray as xr
import cubed 

spec = cubed.Spec(
    work_dir="s3://cmy-bucket/cubed/scratch",
    allowed_mem="2GB",
    executor_name="coiled",
    executor_options={'coiled_kwargs':{'vm_type':'m8g.large'}}
)

ds = xr.tutorial.open_dataset(
    "air_temperature", chunked_array_type="cubed", from_array_kwargs={"spec": spec} 
)
del ds.air.encoding['chunks']

rds = ds.chunk({'time':1}, chunked_array_type="cubed",from_array_kwargs={"spec": spec} )
rds.to_zarr("s3://my-bucket/cubed/air.zarr", mode="w", consolidated=False)

The cubed docs are looking great by the way!

norlandrhagen avatar Sep 12 '25 18:09 norlandrhagen

Edit: Adding from_array_kwargs={"spec": spec} seemed to fix it!

Glad you got it fixed @norlandrhagen!

This has come up before (https://github.com/cubed-dev/cubed/issues/539#issuecomment-2270208349), but it is still a problem.

The cubed docs are looking great by the way!

Thanks!

I think we could improve the docs by mentioning that if you have a Spec object you need to pass it to the Xarray open_dataset and chunk functions. The examples assume that you have a YAML file (which is picked up globally), but that isn't always the case.

tomwhite avatar Sep 15 '25 08:09 tomwhite