xarray icon indicating copy to clipboard operation
xarray copied to clipboard

xarray allows several types for netcdf attributes. Is it expected ?

Open ghislainp opened this issue 3 years ago • 3 comments

What is your issue?

Xarray is permissive regarding the type of the attributes. If using a wrong type, the error reveals the valid types: For serialization to netCDF files, its value must be of one of the following types: str, Number, ndarray, number, list, tuple

Using a non iterable type used to raise an Exception when reading the saved netcdf, but this is now solved with #7085

The pending question is whether it is valid to save netcdf attributes with type other than a string or not. The following lines are working (in a notebook):

xr.DataArray([1, 2, 3], attrs={'units': 1}, name='x').to_netcdf("tmp.nc")
!ncdump tmp.nc

xr.DataArray([1, 2, 3], attrs={'units': np.nan}, name='x').to_netcdf("tmp.nc")
!ncdump tmp.nc

xr.DataArray([1, 2, 3], attrs={'units': ['xarray', 'is', 'very', 'permissive', ]}, name='x').to_netcdf("tmp.nc")
!ncdump tmp.nc

On the other hand, the following line raises an error:

xr.DataArray([1, 2, 3], attrs={'units': None}, name='x').to_netcdf("tmp.nc")

ghislainp avatar Sep 27 '22 20:09 ghislainp

What exactly do you mean by netcdf attributes? The special meaning ones like units, _FillValue or long_name?

In general xarray is quite relaxed on what users put into the attrs, internally it is simply a dict of anything (see e.g. #7111 where people put in recursive DataArrays).

As soon as you try to write it into a netCDF it has to be serializable, but that's why it raises an error.

Personally I think the current behavior is fine (ofc, plotting etc should be able to deal with non-standard units etc.)

headtr1ck avatar Oct 02 '22 06:10 headtr1ck

  • this issue was submitted because of https://github.com/pydata/xarray/pull/7085#discussion_r981443796
  • I also tend to agree that this behavior is fine for the non-specific netcdf attributes.
  • For specific attributes as unit and _FillValue, is it also fine ? I would expect that Dataset.to_netcdf check the type (string for unit and the type of the variable for _FillValue and raise at least a warning. It is currently possible to save a number for the unit... Is the resulting netcdf CF-compliant ?
  • if any kind of types for attributes is allowed, this requires to chase bugs of the kind solved in #7085.

ghislainp avatar Oct 02 '22 09:10 ghislainp

I believe NetCDF, CF-Conventions, and xarray should all be considered independently. It is not necessarily the responsibility of xarray serialization to ensure a CF-complaint file...

acrosby avatar Oct 04 '22 20:10 acrosby