satpy icon indicating copy to clipboard operation
satpy copied to clipboard

cf-writer failing with h5netcdf as engine for some versions of h5py

Open ninahakansson opened this issue 4 years ago • 2 comments

Describe the bug The wavelength attribute causes the cf writer to crash when using the h5netcdf-engine for some versions of h5py.

To Reproduce

# Your code here
from satpy import Scene
import xarray as xr
from satpy.dataset.dataid import WavelengthRange
scn = Scene()
wl = WavelengthRange(min=10.3, central=10.8, max=11.3, unit='µm')
scn['test-array'] = xr.DataArray(
    [1, 2, 3],
    attrs={'wavelength': wl})
scn.save_datasets(filename='temp_test.nc', writer='cf', engine='h5netcdf')

Expected behavior No errors and that a file be saved.

Actual results Text output of actual results or error messages including full tracebacks if applicable.

Traceback (most recent call last):
  File "/home/a001865/git/foua_python_scripts/nina/test_satpy_wavelength.py", line 11, in <module>
    scn.save_datasets(filename='temp_test.nc', writer='cf', engine='h5netcdf')
  File "/home/a001865/git/satpy/satpy/scene.py", line 1373, in save_datasets
    return writer.save_datasets(datasets, compute=compute, **save_kwargs)
  File "/home/a001865/git/satpy/satpy/writers/cf_writer.py", line 717, in save_datasets
    **other_to_netcdf_kwargs)
  File "/home/a001865/opt/CONDA_JUNE/envs/PPSv2018pCMSAFOct2020_Py3/lib/python3.6/site-packages/xarray/core/dataset.py", line 1567, in to_netcdf
    invalid_netcdf=invalid_netcdf,
  File "/home/a001865/opt/CONDA_JUNE/envs/PPSv2018pCMSAFOct2020_Py3/lib/python3.6/site-packages/xarray/backends/api.py", line 1082, in to_netcdf
    dataset, store, writer, encoding=encoding, unlimited_dims=unlimited_dims
  File "/home/a001865/opt/CONDA_JUNE/envs/PPSv2018pCMSAFOct2020_Py3/lib/python3.6/site-packages/xarray/backends/api.py", line 1128, in dump_to_store
    store.store(variables, attrs, check_encoding, writer, unlimited_dims=unlimited_dims)
  File "/home/a001865/opt/CONDA_JUNE/envs/PPSv2018pCMSAFOct2020_Py3/lib/python3.6/site-packages/xarray/backends/common.py", line 255, in store
    variables, check_encoding_set, writer, unlimited_dims=unlimited_dims
  File "/home/a001865/opt/CONDA_JUNE/envs/PPSv2018pCMSAFOct2020_Py3/lib/python3.6/site-packages/xarray/backends/common.py", line 293, in set_variables
    name, v, check, unlimited_dims=unlimited_dims
  File "/home/a001865/opt/CONDA_JUNE/envs/PPSv2018pCMSAFOct2020_Py3/lib/python3.6/site-packages/xarray/backends/h5netcdf_.py", line 294, in prepare_variable
    nc4_var.attrs[k] = v
  File "/home/a001865/opt/CONDA_JUNE/envs/PPSv2018pCMSAFOct2020_Py3/lib/python3.6/site-packages/h5netcdf/attrs.py", line 30, in __setitem__
    self._h5attrs[key] = value
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/home/a001865/opt/CONDA_JUNE/envs/PPSv2018pCMSAFOct2020_Py3/lib/python3.6/site-packages/h5py/_hl/attrs.py", line 103, in __setitem__
    self.create(name, data=value)
  File "/home/a001865/opt/CONDA_JUNE/envs/PPSv2018pCMSAFOct2020_Py3/lib/python3.6/site-packages/h5py/_hl/attrs.py", line 180, in create
    htype = h5t.py_create(original_dtype, logical=True)
  File "h5py/h5t.pyx", line 1629, in h5py.h5t.py_create
  File "h5py/h5t.pyx", line 1653, in h5py.h5t.py_create
  File "h5py/h5t.pyx", line 1719, in h5py.h5t.py_create
TypeError: No conversion path for dtype: dtype('<U32')

Environment Info:

  • OS: Linux
  • Satpy Version: 0.24.0
  • h5py: 3.1.0
  • h5netcdf: 0.8.1

Additional context

There are no problem if I use these combination of versions: h5py: 2.10.0 and h5netcdf 0.8.0 h5py: 2.10.0 and h5netcdf 0.8.1

ninahakansson avatar Nov 18 '20 10:11 ninahakansson

FYI: Can't reproduce with h5py 3.7.0 and h5netcdf 1.0.1.

kmuehlbauer avatar Jun 30 '22 07:06 kmuehlbauer

Given the age of this and @kmuehlbauer telling us it doesn't show up anymore, maybe we can close this @ninahakansson ?

djhoese avatar Jun 30 '22 14:06 djhoese