pymech
pymech copied to clipboard
eX-Mech
Xarray's open_mfdataset function does not work without dask: `ValueError: unrecognized chunk manager dask - must be one of: []`
For example see this test case tests/test_mfdataset.py::test_xr_open_mfdataset
>>> import pymech as pm
>>> pm.open_dataset("/tmp/channel3D_0.f00001")
<xarray.Dataset> Size: 15MB
Dimensions: (z: 64, y: 64, x: 64)
Coordinates:
* x (x) float64 512B 0.0 0.05037 0.1603 0.3105 ... 6.123 6.233 6.283
* y (y) float64 512B -1.0 -0.9974 -0.9918 ... 0.9918 0.9974 1.0
* z (z) float64 512B 0.0 0.02518 0.08017 0.1553 ... 3.061 3.116 3.142
time float64 8B 0.2
Data variables:
xmesh (z, y, x) float64 2MB 0.0 0.05037 0.1603 ... 6.123 6.233 6.283
ymesh (z, y, x) float64 2MB -1.0 -1.0 -1.0 -1.0 -1.0 ... 1.0 1.0 1.0 1.0
zmesh (z, y, x) float64 2MB 0.0 0.0 0.0 0.0 ... 3.142 3.142 3.142 3.142
ux (z, y, x) float64 2MB 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0 0.0
uy (z, y, x) float64 2MB 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0 0.0
uz (z, y, x) float64 2MB 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0 0.0
pressure (z, y, x) float64 2MB 0.004913 0.0467 0.0195 ... 0.03768 0.0761
>>> ds = pm.open_dataset("/tmp/channel3D_0.f00001")
>>> import xarray as xr
>>> xr.open_dataset("/tmp/channel3D_0.f00001", engine="pymech")
<xarray.Dataset> Size: 15MB
Dimensions: (z: 64, y: 64, x: 64)
Coordinates:
* x (x) float64 512B 0.0 0.05037 0.1603 0.3105 ... 6.123 6.233 6.283
* y (y) float64 512B -1.0 -0.9974 -0.9918 ... 0.9918 0.9974 1.0
* z (z) float64 512B 0.0 0.02518 0.08017 0.1553 ... 3.061 3.116 3.142
time float64 8B ...
Data variables:
xmesh (z, y, x) float64 2MB ...
ymesh (z, y, x) float64 2MB ...
zmesh (z, y, x) float64 2MB ...
ux (z, y, x) float64 2MB ...
uy (z, y, x) float64 2MB ...
uz (z, y, x) float64 2MB ...
pressure (z, y, x) float64 2MB ...
>>> xr.open_mfdataset("/tmp/channel3D_0.f00001", engine="pymech")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/avmo/Sources/snek5000/pymech/.nox/release-tests-dist_type-only-binary/lib/python3.9/site-packages/xarray/backends/api.py", line 1077, in open_mfdataset
datasets = [open_(p, **open_kwargs) for p in paths]
File "/home/avmo/Sources/snek5000/pymech/.nox/release-tests-dist_type-only-binary/lib/python3.9/site-packages/xarray/backends/api.py", line 1077, in <listcomp>
datasets = [open_(p, **open_kwargs) for p in paths]
File "/home/avmo/Sources/snek5000/pymech/.nox/release-tests-dist_type-only-binary/lib/python3.9/site-packages/xarray/backends/api.py", line 594, in open_dataset
ds = _dataset_from_backend_dataset(
File "/home/avmo/Sources/snek5000/pymech/.nox/release-tests-dist_type-only-binary/lib/python3.9/site-packages/xarray/backends/api.py", line 370, in _dataset_from_backend_dataset
ds = _chunk_ds(
File "/home/avmo/Sources/snek5000/pymech/.nox/release-tests-dist_type-only-binary/lib/python3.9/site-packages/xarray/backends/api.py", line 318, in _chunk_ds
chunkmanager = guess_chunkmanager(chunked_array_type)
File "/home/avmo/Sources/snek5000/pymech/.nox/release-tests-dist_type-only-binary/lib/python3.9/site-packages/xarray/namedarray/parallelcompat.py", line 117, in guess_chunkmanager
raise ValueError(
ValueError: unrecognized chunk manager dask - must be one of: []
Further reading:
- https://github.com/pydata/xarray/issues/7856
- https://stackoverflow.com/questions/76409916/python-xarray-valueerror-unrecognized-chunk-manager-dask-must-be-one-of
We need to check if this is an upstream issue or the way we implement pymech as a xarray backend.
The PR to improve the situtation with xarray landed on v.2025.01.0 (Jan 3, 2025) with https://github.com/pydata/xarray/pull/9676