intake-xarray icon indicating copy to clipboard operation
intake-xarray copied to clipboard

How to template chunk specs?

Open willirath opened this issue 5 years ago • 2 comments

I have a use case where I'd like to catalog NEMO data and chunk depth dims which are called differently (deptht, depthu, ...).

The following explains (I hope), what I want to achieve:

[...]
sources:
  NEMO-Exp01:
    description: NEMO-Exp01
    driver: netcdf
    parameters:
      grid:
        description: grid
        allowed: ["t", "u", "v", "w"]
        default: "t"
        type: str
    args:
      urlpath: '/data/NEMO-Exp01/NEMO-Exp01_1m_20010101_20011231_grid_{{ grid.upper() }}.nc'
      chunks: {'depth{{ grid }}': 1, }
[...]

Is there a short way of doing this (without having 34 different entries that only differ in their grid)?

willirath avatar Dec 17 '19 07:12 willirath

What does the second entry looks like? If you don't want to write out all of the entries, you could generate them or even write a Catalog class which can generate the entries at runtime.

martindurant avatar Dec 20 '19 14:12 martindurant

What does the second entry looks like?

Not sure I under stand. The full chunk spec would be something like:

{
    'time': 1,
    'depth{{ grid }}': 1,
    'y': None,
    'x': 200
}

If you don't want to write out all of the entries, you could generate them or even write a Catalog class which can generate the entries at runtime.

I guess generating them is the way to go then.

Should I close here?

willirath avatar Dec 20 '19 17:12 willirath