incorrect latlons from navgem grib files

[abrammer]

Getting some erroneous values back from reading in latlons() from navgem files (available on nomads). All the other variables seem to be scaled and read correctly, just something going awry with the lat lon information.

e.g. https://nomads.ncep.noaa.gov/pub/data/nccf/com/fnmoc/prod/navgem.20190822/navgem_2019082200f000.grib2 Update the date in the url if it's cycled off the nomads server.

The latlons returned are scaled by something strange.

>>> import pygrib
>>> pg = pygrib.open("navgem_2019082200f000.grib2")
>>> msg = pg.select(shortName='u', level=850)[0]
>>> msg.latlons()
(array([[-1373.3119707 , -1373.3119707 , -1373.3119707 , ...,
        -1373.3119707 , -1373.3119707 , -1373.3119707 ],
       [-1365.68245975, -1365.68245975, -1365.68245975, ...,
        -1365.68245975, -1365.68245975, -1365.68245975],
       [-1358.05294881, -1358.05294881, -1358.05294881, ...,
        -1358.05294881, -1358.05294881, -1358.05294881],
       ...,
       [ 1358.05294881,  1358.05294881,  1358.05294881, ...,
         1358.05294881,  1358.05294881,  1358.05294881],
       [ 1365.68245975,  1365.68245975,  1365.68245975, ...,
         1365.68245975,  1365.68245975,  1365.68245975],
       [ 1373.3119707 ,  1373.3119707 ,  1373.3119707 , ...,
         1373.3119707 ,  1373.3119707 ,  1373.3119707 ]]), array([[-360.        , -352.37048905, -344.7409781 , ..., 5110.35934997,
        5117.98886091, 5125.61837186],
       [-360.        , -352.37048905, -344.7409781 , ..., 5110.35934997,
        5117.98886091, 5125.61837186],
       [-360.        , -352.37048905, -344.7409781 , ..., 5110.35934997,
        5117.98886091, 5125.61837186],
       ...,
       [-360.        , -352.37048905, -344.7409781 , ..., 5110.35934997,
        5117.98886091, 5125.61837186],
       [-360.        , -352.37048905, -344.7409781 , ..., 5110.35934997,
        5117.98886091, 5125.61837186],
       [-360.        , -352.37048905, -344.7409781 , ..., 5110.35934997,
        5117.98886091, 5125.61837186]]))

Looking at the pygrib code it looks like it's probably related to funky values here:

>>> msg.longitudeOfFirstGridPointInDegrees
0.0
>>> msg.longitudeOfLastGridPointInDegrees
5485.618371862363
>>> msg.latitudeOfFirstGridPointInDegrees
-1373.3119707026779
>>> msg.latitudeOfLastGridPointInDegrees
1373.3119707026779

Are these being unpacked incorrectly or is there an error in the grib file? We have a work around for manually scaling the values back to what they should be but fixing a bug is always better than working around it.

Edit: of course right after submitting this I found #102 and I'm still on pygrib. 2.0.3. Will install the new release and update. Updated to 2.0.4 through conda and the above still holds true.

[abrammer]

Coming back to this, I tracked this down to most likely an incorrect GRIB file. subdivisionsOfBasicAngle from Section 3, template 0, is encoded as all 1s for a 16bit int instead of a 32bit int. It should be treated as missing but instead is passed through as (2**16)-1 == 65535.

In ncepgrib2.py this is a fairly easy fix, we can add a check at https://github.com/jswhit/pygrib/blob/b16e444360f1c07489814c8ef8f5c70effcdadc1/ncepgrib2.py#L482 for if divisor <= 0 or divisor == 65535: divisor = 1.e6 and assume that no one is going to use a scale factor of 66535 legitimately. For the eccodes version it seems more difficult as all this decoding is done with eccodes.

Or better solution, NRL fixes their currently broken GRIB file.