PyPSA
ERA5 CDS requests which return a mixture of ERA5 and ERA5T data
Description
As reported here, CDS will return an dataset with an extra expver dimension when a call spans over an date range that consist of both ERA5 and preliminary ERA5T data. This is not handled in Atlite.
expver 1 : ERA5 data, that has been quality checked.
expver 5 : ERA5T preliminary data (the three last months).
Expected Behavior
The expver dimension should be removed.
A bonus would be to report the date when the ERA5T (expver 5) data starts as this data might get updated/corrected later.
Fix
An easy fix is that has worked in all my use cases:
cutout.data = cutout.data.reduce(np.nansum, 'expver')
Guess it could be done somewhere earlier in the preparation of the data, i.e. before any derived variables are calculated. Having the extra expver dimension also doubles the memory footprint, so it can be good to be thoughtful about this also when/if splitting cds calls in the time dimension
