Cubed with coiled executor example clarification
Hey there @tomwhite,
I was interesting in trying cubed for rechunking using the coiled executor. This works fine, but ~~I don't see any coiled functions created in the coiled dashboard. Wondering if there is some additional config I need?~~
Edit: Adding from_array_kwargs={"spec": spec} seemed to fix it!
MRE
# cubed_version: '0.22.0'
# coiled_version: '1.120.0'
# xarray_version: '2025.9.0'
import xarray as xr
import cubed
spec = cubed.Spec(
work_dir="s3://cmy-bucket/cubed/scratch",
allowed_mem="2GB",
executor_name="coiled",
executor_options={'coiled_kwargs':{'vm_type':'m8g.large'}}
)
ds = xr.tutorial.open_dataset(
"air_temperature", chunked_array_type="cubed", from_array_kwargs={"spec": spec}
)
del ds.air.encoding['chunks']
rds = ds.chunk({'time':1}, chunked_array_type="cubed",from_array_kwargs={"spec": spec} )
rds.to_zarr("s3://my-bucket/cubed/air.zarr", mode="w", consolidated=False)
The cubed docs are looking great by the way!
Edit: Adding
from_array_kwargs={"spec": spec}seemed to fix it!
Glad you got it fixed @norlandrhagen!
This has come up before (https://github.com/cubed-dev/cubed/issues/539#issuecomment-2270208349), but it is still a problem.
The cubed docs are looking great by the way!
Thanks!
I think we could improve the docs by mentioning that if you have a Spec object you need to pass it to the Xarray open_dataset and chunk functions. The examples assume that you have a YAML file (which is picked up globally), but that isn't always the case.