thredds icon indicating copy to clipboard operation
thredds copied to clipboard

CF timeSeries featureType and forecast_reference_time don't play nice together.

Open dblodgett-usgs opened this issue 7 years ago • 5 comments

I've been working with some National Water Model data pulled straight off the NCEP feed. Trying to build daily forecast collection files that I'll then glue together along "reference_time" to create stand alone THREDDS services that can be queried like an FMRC.

Have been experimenting with the FMRC aggregation and ran into too many road blocks, so have just been going nco / straight daily NetCDF files for now.

The issue is NetCDF Java (toolsUI) pointFeature isn't recognizing the data once it has a forecast dimension. Thinking about the use cases and applications of the pointFeature that I'm aware of, I guess I'm not surprised that this isn't handled, but figured I'd leave a record that this is something that maybe should work.

This works:

netcdf test_one {
dimensions:
	time = 80 ;
	feature_id = 2729077 ;
variables:
	float streamflow(time, feature_id) ;
		streamflow:long_name = "River Flow" ;
		streamflow:missing_value = -999900 ;
		streamflow:units = "m3 s-1" ;
		streamflow:coordinates = "longitude latitude time " ;
	int time(time) ;
		time:long_name = "valid output time" ;
		time:standard_name = "time" ;
		time:units = "minutes since 1970-01-01 00:00:00 UTC" ;
		time:axis = "T" ;
	int feature_id(feature_id) ;
		feature_id:long_name = "Reach ID" ;
		feature_id:comment = "NHDPlusv2 ComIDs within CONUS, arbitrary Reach IDs outside of CONUS" ;
		feature_id:cf_role = "timeseries_id" ;
	int reference_time ;
		reference_time:long_name = "model initialization time" ;
		reference_time:standard_name = "forecast_reference_time" ;
		reference_time:units = "minutes since 1970-01-01 00:00:00 UTC" ;
	float latitude(feature_id) ;
		latitude:units = "degrees_north" ;
		latitude:long_name = "latitude of point feature" ;
		latitude:standard_name = "latitude" ;
		latitude:axis = "Y" ;
	float longitude(feature_id) ;
		longitude:units = "degrees_east" ;
		longitude:long_name = "longitude of point feature" ;
		longitude:standard_name = "longitude" ;
		longitude:axis = "X" ;

// global attributes:
		:featureType = "timeSeries" ;
		:proj4 = "+proj=longlat +datum=NAD83 +no_defs" ;
		:esri_pe_string = "GEOGCS[GCS_North_American_1983,DATUM[D_North_American_1983,SPHEROID[GRS_1980,6378137.0,298.257222101]],PRIMEM[Greenwich,0.0],UNIT[Degree,0.017453292519943295]]" ;
		:title = "NWM Forcast Model Run Collection" ;
		:Conventions = "CF-1.7" ;
}

This doesn't work:

netcdf test_two {
dimensions:
	reference_time = UNLIMITED ; // (1 currently)
	time = 80 ;
	feature_id = 2729077 ;
variables:
	float streamflow(reference_time, time, feature_id) ;
		streamflow:long_name = "River Flow" ;
		streamflow:missing_value = -999900 ;
		streamflow:units = "m3 s-1" ;
		streamflow:coordinates = "longitude latitude time reference_time" ;
	int time(reference_time, time) ;
		time:long_name = "valid output time" ;
		time:standard_name = "time" ;
		time:units = "minutes since 1970-01-01 00:00:00 UTC" ;
		time:axis = "T" ;
	int feature_id(feature_id) ;
		feature_id:long_name = "Reach ID" ;
		feature_id:comment = "NHDPlusv2 ComIDs within CONUS, arbitrary Reach IDs outside of CONUS" ;
		feature_id:cf_role = "timeseries_id" ;
	int reference_time(reference_time) ;
		reference_time:long_name = "model initialization time" ;
		reference_time:standard_name = "forecast_reference_time" ;
		reference_time:units = "minutes since 1970-01-01 00:00:00 UTC" ;
	float latitude(feature_id) ;
		latitude:units = "degrees_north" ;
		latitude:long_name = "latitude of point feature" ;
		latitude:standard_name = "latitude" ;
		latitude:axis = "Y" ;
	float longitude(feature_id) ;
		longitude:units = "degrees_east" ;
		longitude:long_name = "longitude of point feature" ;
		longitude:standard_name = "longitude" ;
		longitude:axis = "X" ;

// global attributes:
		:featureType = "timeSeries" ;
		:proj4 = "+proj=longlat +datum=NAD83 +no_defs" ;
		:esri_pe_string = "GEOGCS[GCS_North_American_1983,DATUM[D_North_American_1983,SPHEROID[GRS_1980,6378137.0,298.257222101]],PRIMEM[Greenwich,0.0],UNIT[Degree,0.017453292519943295]]" ;
		:title = "NWM Forcast Model Run Collection" ;
		:Conventions = "CF-1.7" ;
}

dblodgett-usgs avatar Apr 03 '18 13:04 dblodgett-usgs

Yes, 2D time probably should work for DGSs, but bear in mind that your dataset is no longer CF-compliant once you add the reference_time dimension: http://cfconventions.org/Data/cf-conventions/cf-conventions-1.7/cf-conventions.html#_features_and_feature_types. So ,the deficiency is larger than just NetCDF-Java.

cwardgar avatar Apr 03 '18 14:04 cwardgar

Interesting... I wouldn't read the DSG spec as disallowing 2D time, but I can see how it could be interpreted that way. Thanks. I mostly wanted to leave a record of this and make sure I wasn't missing something. Will close this and bring it back up in the future if it becomes a strict need. -- really not clear how reference_time would play in the ncSOS API and that's really the potential value here IMHO.

dblodgett-usgs avatar Apr 03 '18 15:04 dblodgett-usgs

@dblodgett-usgs I'm agree with you about DSG spec it's more general than the implicit interpretation been made when examples, in the CF, are exposed.

cofinoa avatar Apr 03 '18 16:04 cofinoa

@dblodgett-usgs - mind if we leave this issue open? This is something we'd want to address in the future. It'd also be good to ping the CF list to get some clarification on the case of a 2D time coordinate for a DSG.

lesserwhirls avatar Apr 03 '18 17:04 lesserwhirls

Do I mind? Not one bit! Just didn't want to muddy your issue list with open ended requests!

dblodgett-usgs avatar Apr 03 '18 17:04 dblodgett-usgs