Have filenames in streams use years from $DATM_YR* rather than hardcoded min/max

[ekluzek]

trafficstars

This is especially true for datm because the list of files is long. But, it could apply to other data models as well. Currently the minimum and maximum years for the filenames in streams is hardcoded in. The actual years that will be used is set by $DATM_YR* variables. But, for instance for CLMGSWP3v1.Solar you have this...

<file first_year="1901" last_year="2014">$DIN_LOC_ROOT_CLMFORC/atm_forcing.datm7.GSWP3.0.5d.v1.c170516/Solar/clmforc.GSWP3.c2011.0.5x0.5.Solr.%ym.nc</file>

The downside I see to this is that at initialization it will open each of the files in the list in turn and get the time information from them. This can take awhile so it seems more prudent to have the list shortened so that it only has the filenames of the years it's actually going to use.

It will also make a difference when downloading the files. If the full range is given it will download the full list, even if the user wants one or two years. I actually recommend to users that they use $DATM_YR* to get a shorter list of files using check_input_data --download, and then after they've been downloaded to increase it to download another section.

[ekluzek]

The one catch to this is that I think in some cases the previous file (before the starting year) and the next file (after the ending year) is needed. I'm not quite sure when that's required though.

I also see the value in having the data range expressed in the file. But, maybe that could be a separate variable. And if that was done you could have the streams mechanism check if it was in range. I don't consider that important though because you get the same information when you try to download files outside the valid period where data exists.

[ekluzek]

I did notice many streams that have hardcoded values for stream_year_* settings. And I wonder if they shouldn't be changed to operate on $DATM_YR* as well, although it depends on the specific stream.

[ekluzek]

OK, I'm NOT doing this as it's unclear if it should be done. This needs some discussion. With more testing I found out that it will limit the years in the list based on BOTH $DATM_YR_* and the min and max in the streams file. It doesn't declare an error though if your DATM_YR_START or DATM_YR_END is out of range -- it'll let that go, but set it up for those years without the needed files. This means you'll get an error when you run the model rather than at preview_namelist time. So I think it should complain and die if that's the case.