netcdf-java icon indicating copy to clipboard operation
netcdf-java copied to clipboard

Incorrect times and dataset classification for "observational" GRIB datasets

Open JohnLCaron opened this issue 3 years ago • 2 comments

GRIB Collection MRMS_NLDN are being classified as MRC, should be MRUTC. (May be others)

  • also occasionally MRMS_BaseReflectivity

MRMS_NLDN is an "observational dataset" where forecast time == reference time always. The Reference time has hour/min/second fields But Forecast time is "0 minutes since reference", which makes CDM use minutes as the reference and forecast time unit. So when time between reference times < minute, get diff == 0, which triggers "not unique time coordinates".

CoordinateTime2D.hasUniqueTimes() is where that fails, but the issue is in Grib1or2CollectionBuilder, eg at line 289 in Grib2, where the VariableBag.timeUnit is passed to

    CoordinateTime2D.Builder2 builder2D = new CoordinateTime2D.Builder2(isTimeInterval, cust, vb.timeUnit, code);

which makes CoordinateRuntime use that timeUnit.

This also means the reference dates are wrong, eg:

2020-10-27T00:00:01Z
 2020-10-27T00:01:01Z
 2020-10-27T00:02:01Z
 2020-10-27T00:03:01Z
 2020-10-27T00:04:01Z
 2020-10-27T00:05:01Z
 2020-10-27T00:06:01Z
  ...

instead of

Run Times: reftime1 (minutes since 2020-10-27T00:00:01Z)
   2020-10-27T00:00:01Z (0.000000)
   2020-10-27T00:02Z (1.983333)
   2020-10-27T00:02:21Z (2.333333)
   2020-10-27T00:04Z (3.983333)
   2020-10-27T00:04:11Z (4.166667)
   2020-10-27T00:05:13Z (5.200000)
   2020-10-27T00:06:38Z (6.616667)
  ...

Possible ways to fix:

  1. Always use seconds as the time unit.
  2. Put a check into CoordinateRuntime to look for units that are incompatible with the actual values. Change to seconds if needed.
  3. Have the user configure the collection as an "observation collection", which uses seconds in its reference date, and checks that forecast = reference.

Note that we dont need to use millisecs since thats not possible in GRIB.

This effects all versions since the big bang. Need to fix and regenerate the ncx4.

JohnLCaron avatar Jun 18 '21 19:06 JohnLCaron

Fixed by PR #727

JohnLCaron avatar Jun 29 '21 00:06 JohnLCaron

Well, could be backported, so Ill leave open.

JohnLCaron avatar Jun 29 '21 01:06 JohnLCaron