atlite icon indicating copy to clipboard operation
atlite copied to clipboard

Data type error when building cutout with SARAH v3

Open PeterKlein11 opened this issue 2 years ago • 3 comments

Description

The latest SARAH v3 downloads for SID and SIS include a few extra data variables in the xarray dataset, e.g record_status which is an int, which breaks the interpolation function.

AssertionError: interpolate only supports datasets with homogeneous dtype

Simple fix is to remove from dataset, and then it seems to run fine. Not sure if needed for anything else?

PeterKlein11 avatar Jan 10 '24 17:01 PeterKlein11

Thanks for reporting.

Can you share the link from where you downloaded the SARAH v3 data? Without knowing what record_status details exactly, it would probably be the easiest approach to just drop that variable before interpolation.

One should also check whether there were other changes from SARAHv2 to SARAHv3 in the variables and conventions. If not, then the fix would be as simple as you have described it.

We'd be happy to receive a PR if you feel like it :)

euronion avatar Jan 11 '24 07:01 euronion

Here are some details on record_status, extracted with xarray from a SARAH-3 netCDF file:

comment :
    Overall status of each record (timestamp) in this file. If a record is flagged as not ok, it is recommended not to use it.
flag_meanings :
    ok void bad_quality
long_name :
    Record Status
flag_values :
    [0 1 2]

ladder-upstroke avatar Mar 05 '24 13:03 ladder-upstroke

Thanks.

Is the status normally 0 for all records? Or does a dataset usually have a mix of ok, void and bad_quality entries?

Did something similar exist before and we just ignored it, or is this a new variable that was necessary to be introduced with the changes to SARAHv3?

If the field should usually be 0 , we could check if that's the case for all entries and in that case drop the variable for further processing.

Else: What happens if you convert the record_status to a floating number?

euronion avatar Mar 05 '24 23:03 euronion

@PeterKlein11 This has probably been resolved with #352

euronion avatar Jul 11 '24 23:07 euronion