Wflow.jl icon indicating copy to clipboard operation
Wflow.jl copied to clipboard

Improve error when _FillValue in time dimension

Open JoostBuitink opened this issue 2 years ago • 3 comments

Throw an error when _FillValue is present in the time dimension, as Wflow is unable to run with this attribute present. Currently, an error is thrown in the update_forcing! function, at the following line https://github.com/Deltares/Wflow.jl/blob/0e0b50bbc69ca84ea910ea8637a19f8a21e18d1d/src/io.jl#L270

Current workaround (with Python, xarray): set ds.time.encoding["_FillValue"] = None prior to writing the netcdf file. See also https://github.com/Deltares/hydromt_wflow/issues/60

JoostBuitink avatar Sep 27 '22 15:09 JoostBuitink

So it currently throws an error when the time dimension has _FillValue, even if there are no actual missing times? Or only if there are missing times?

Probably we can also just support this rather than throwing an error. We'd need to handle missings explicitly in https://github.com/Deltares/Wflow.jl/blob/0e0b50bbc69ca84ea910ea8637a19f8a21e18d1d/src/io.jl#L162-L174.

We'd probably have to widen the type signature for times, and use skipmissing:

julia> findfirst(>=(3), skipmissing([missing, 2, 3, missing]))
3

visr avatar Oct 06 '22 21:10 visr

So it currently throws an error when the time dimension has _FillValue, even if there are no actual missing times? Or only if there are missing times?

In this case an error was thrown with a _FillValue in the time dimension, without missing times. Missing data is not allowed in coordinate variables according to NetCDF metadata conventions, so I think it is preferred to throw an error instead of supporting this in Wflow.

verseve avatar Oct 07 '22 08:10 verseve

Ok, yes throwing an error is definitely defensible.

Related, right now we do accept _FillValue without actual missing in the spatial coordinate dimensions, and hydromt_wflow writes it with _FillValue currently: https://github.com/Deltares/hydromt_wflow/issues/60#issuecomment-1216508822

visr avatar Oct 07 '22 08:10 visr