Add MPAS 480, 120, 60, 30, 15-3-conus surface files

[MiCurry]

Description of changes

This merge adds the surface data and landuse.timeseries files for the CAM-MPAS dycore. At present, these grids are:

• mpasa480 - 480 km quasi-uniform, global with 2,462 horizontal grid columns
• mpasa120 - 120 km quasi-uniform, global with 40,962 horizontal grid columns
• mpasa60 - 60 km quasi-uniform, global with 163,842 horizontal grid columns
• mpasa30 - 30 km quasi-uniform, global with 655362 horizontal grid columns
• mpasa15-3-conus - 15-3 km variable resolution mesh from 15 km that refines down to 3 km. The 3 km region is circular and spans roughly 60 degrees of latitude/latitude. The refinement region is centered over CONUS. This grid has 6,488,066 horizontal grid columns.

Specific notes

Currently, the mpasa480 landuse.timeseries file is not included in this PR. As well, this merge currently does not contain any of the mapping files that are created with mksurfdata_map and will be added shortly.

Contributors other than yourself, if any: @jtruesdal

CTSM Issues Fixed (include github issue #): #1313

Are answers expected to change (and if so in what way)?

No.

Any User Interface Changes (namelist or namelist defaults changes)?

New namelist options added. valid_values for res innamelist_definition_ctsm.xml now include mpasa480, mpasa120, mpasa60, mpasa30, and mpasa15-3.

Testing performed, if any: Testing by hand with CAM/CESM by running F2000climo. Currently, the 60 km grid does not run successfully, but the cause of this may be the CAM topography file.

[MiCurry]

@ekluzek, @jtruesdal I am a little confused with adding the mapping files as I'm seeing a different number for each grid (and a different number than what was created with mksurfdata_map). For example, the 1x1_brazil has 9 grids:

https://github.com/ESCOMP/CTSM/blob/e9c027c6250f3d02518f3469e7d89f92690d1c45/bld/namelist_files/namelist_defaults_ctsm.xml#L1859-L1874

But our mpasa480 grid has 20 (in /glade/p/cesmdata/inputdata/lnd/clm2/mappingdata/maps/mpasa480):

map_0.5x0.5_AVHRR_to_mpasa480_nomask_aave_da_c200930.nc
map_0.5x0.5_MODIS_to_mpasa480_nomask_aave_da_c200930.nc
map_0.5x0.5_TO_mpasa480_aave.200930.nc
map_0.9x1.25_GRDC_to_mpasa480_nomask_aave_da_c200930.nc
map_0.25x0.25_MODIS_to_mpasa480_nomask_aave_da_c200930.nc
map_1km-merge-10min_HYDRO1K-merge-nomask_to_mpasa480_nomask_aave_da_c200930.nc
map_3x3min_GLOBE-Gardner-mergeGIS_to_mpasa480_nomask_aave_da_c200930.nc
map_3x3min_GLOBE-Gardner_to_mpasa480_nomask_aave_da_c200930.nc
map_3x3min_LandScan2004_to_mpasa480_nomask_aave_da_c200930.nc
map_3x3min_MODIS-wCsp_to_mpasa480_nomask_aave_da_c200930.nc
map_3x3min_USGS_to_mpasa480_nomask_aave_da_c200930.nc
map_5x5min_IGBP-GSDP_to_mpasa480_nomask_aave_da_c200930.nc
map_5x5min_ISRIC-WISE_to_mpasa480_nomask_aave_da_c200930.nc
map_5x5min_ORNL-Soil_to_mpasa480_nomask_aave_da_c200930.nc
map_5x5min_nomask_to_mpasa480_nomask_aave_da_c200930.nc
map_10x10min_IGBPmergeICESatGIS_to_mpasa480_nomask_aave_da_c200930.nc
map_10x10min_nomask_to_mpasa480_nomask_aave_da_c200930.nc
map_360x720cru_cruncep_to_mpasa480_nomask_aave_da_c200930.nc
map_mpasa480_nomask_to_0.5x0.5_nomask_aave_da_c200930.nc

I also noticed that I could use createMapEntry.pl, but it only creates 3 entries, for the mpasa480.

Should I just use 1x1_brazil as an example/template and add the same mapping files it does?

[ekluzek]

@MiCurry yes follow the 1x1_brazil example. We used to require mapping files for every grid/land-mask, but now only need a mapping files for the "nomask" mask for each grid. So that cut down on the number of mapping files needed for each supported resolution. But, we haven't removed all the unneeded entries at this point yet. That is something that we should do.

I thought createMapEntry.pl was fixed to do the right thing, but obviously 3 entries is too small, there should be 9 different mapping files for the 9 different grids needed. I might look into that if I get a chance...

[MiCurry]

@MiCurry yes follow the 1x1_brazil example. We used to require mapping files for every grid/land-mask, but now only need a mapping files for the "nomask" mask for each grid. So that cut down on the number of mapping files needed for each supported resolution. But, we haven't removed all the unneeded entries at this point yet. That is something that we should do.

I thought createMapEntry.pl was fixed to do the right thing, but obviously 3 entries is too small, there should be 9 different mapping files for the 9 different grids needed. I might look into that if I get a chance...

@ekluzek sounds great. Perhaps I was using a different version of createMapEntry.pl? Regardless, I think I got enough to start. Thanks!

[MiCurry]

@ekluzek I just pushed an update to the orginal 4 commits (force update) that adds the mapping files using the createMapEntry.pl command. Each grid has only three entries.

When comparing with the 1x1_brazil entries, there are a number of mapping files which are present there, but are not present in the MPAS maps. For instance, there are no grids for the following:

3x3min_nomask_to_mpasaXXX_nomask...
0.9x1.25_nomask_to_mpasaXXX_nomask...
0.25x0.25_nomask_to_mpasaXXX_nomask...
0.125x0.125_nomask_to_mpasaXXX_nomask...

for each of the MPAS grids. It apears that the createMapEntry.pl only grabs grids that are *nomask*_to_*nomask*.

There are also a number of entries, which similarly only have three entries, so I'm a bit confused on the need for nine. As well, We have been running with this branch without references to the mapping grids without problem (i.e. the model does not complain about missing mapping files). Are the mapping files actually needed to actually run the model, or are they just serving as references?

[ekluzek]

@MiCurry there are at least seven mapping files that are definitely needed to create surface datasets.

The needed grids are all no-mask (except 1km), for these grids:

3x3min 5x5min 10x10min 0.5x0l.5 0.25x0.25 0.125x0.125 1km-merge-10min (this is for the HYDRO1K-merge-nomask mask)

If you want to run with VIC hydro you also need a map for 0.9x1.25. I recommend not needing that one, but it's fine to add it to the list.

mkmapdata.sh should have created all of the above maps for you. I think you must have used an older version of mkmapdata so it didn't create all of them for nomask. I'd run it again on your branch, so that it creates all the needed mapping files. createMapEntry.pl only creates entries for files that it finds on disk in inpudata, so if the files are missing there it won't list them (incidentally it also will double list files if there is more than one listed, so you should delete all but the latest entries).

[MiCurry]

I believe I'm seeing a bug with the mkmapdata tool that is on my current branch. The relevant error I'm getting is:

rm: cannot remove 'PET*.Log': No such file or directory
Creating mapping file: map_0.5x0.5_nomask_to_mpasa60_nomask_aave_da_c211011.nc
From input grid: ERROR: unrecognized arguments: scripgriddata
For output grid: /glade/scratch/mcurry/ctsm-test-main/tools/mkmapdata/mpasa60_SCRIP-20210922.nc

Input grid file does NOT exist: ERROR: unrecognized arguments: scripgriddata

Which happens almost immediately after starting regridbatch.sh. The full error message can be found on glade here: /glade/work/mcurry/cesm/ctsm/tools/mkmapdata/regrid.o932613.

I created my grids initially on the ctsm5.1.dev038 tag, and after checking out that branch and running regridbatch.sh, the same maps that are in this commit are created without the above error. I am also seeing this error on the head of ESCOMP:master @ekluzek would you be able to try and recreate the error using the following steps and see if I am missing something?

$ git clone [email protected]:MiCurry/CTSM.git
$ git checkout mpas/grids
$ cd tools/mkmapdata/
$ ln -s /glade/scratch/mcurry/cam-mpas-mapping-domain/mpasa60_redo/mpasa60_SCRIP-20210922.nc .
$ export RES=mpasa60
$ export GRIDFILE="`pwd`/mpasa60_SCRIP-20210922.nc"
$ qsub regridbatch.sh # or ./regridbatch.sh to quickly see the error and not wait in the queue
$ git checkout ctsm5.1.dev038
$ qsub regrdibatch.sh # or ./regridbatch.sh

If you get the same error, the second regrdibatch.sh execution should run and produce grids. If there is something I've done wrong, than please let me know. But the fact that running with ctsm5.1.dev038 runs fine make me think a bug was introduced somewhere.

In the meantime, if you know of a tag that is stable, I think it would be best for me to run with that to create the mapping files I am missing.

[MiCurry]

@ekluzek I am leaving NCAR for a SE position outside of NCAR. @jtruesdal or @mgduda will be taking over this PR. Thus, I will close this PR so one of them can re-open it.

@mgduda and @jtruesdal, I have updated this PR with the relevant surface data maps and the surface datafiles that Erik asked for in this PR.

[ekluzek]

@MiCurry thanks for the heads up and for your work on this. Good luck on your transition away from NCAR. Take care...

[ekluzek]

@jtruesdal and @mgduda what's the status of bringing MPASA surface datasets into CTSM? Should this PR be reopened, with one of you? Or should there be a new PR opened on this?

[jtruesdal]

@ekluzek Yes this should be reopened. I'll discuss with @mgduda. I believe Miles finished most of the work on these. Will you have time to review these sometime in the near future?

[ekluzek]

@jtruesdal that sounds great. I just reopened. I'll make it a draft for now though. Do you have collaborator access to Miles branch? We might need to give you that before proceeding with his branch.

Yes, we will schedule this to come in when you think it's ready. This is actually important to come in before we create all new surface datasets for ctsm5.2.

[jtruesdal]

@ekluzek checked my repositories and I am not a collaborator on the branch. Can you add me as a collaborator?

[ekluzek]

@jtruesdal this is something that @MiCurry needs to do. @MiCurry can you make sure both @jtruesdal and I are collaborators on your github CTSM repository fork? You go to the settings tag, and then collaborators.

If we don't get a response from @MiCurry I could move the branch to my fork (or you could move it to yours) and we could go from there. I don't have any other way to contact Miles, as his ucar email account is closed.

[jtruesdal]

Thanks. We can give it a few days to hear from him, I will start testing with this branch and files.

[mvertens]

@erik @jtruesdal - @MiCurry is no longer at NCAR. The person we need to contact is Michael Duda. Not sure what his github id is.

[MiCurry]

Lucky for y'all I just checked my GitHub notifications this week! @jtruesdal @ekluzek @mvertens I just added @jtruesdal to as a collaborator to my fork if he needs it. Michael Duda's GitHub account is @mgduda.

[ekluzek]

@MiCurry awesome, thanks so much. I'm glad you checked! We didn't have a way to contact you outside of github.

@jtruesdal go ahead and start working on this branch and let me know when it's ready to schedule to come in.

[ekluzek]

We should add at least one test for the MPAS grids. I'm thinking mpasa480 for 2000 as the lowest resolution.

I'm guessing that mpasa120 is the main resolution that's actually used. It looks like CAM does testing for both mpasa480 and mpasa120. So we should add the mpasa120 to the clm_science test list.

[jtruesdal]

@ekluzek If you would like to add tests that's fine although we will be testing this in the atm component. A 9 step 288 core SMS 2000 mpasa120 test will take about 25 minutes to compile/build/run.

[ekluzek]

@jtruesdal I noticed the mpasa15 landuse.timeseries file is 3/4 of a TByte. If you are using it, that's fine, but my suspicion is that you aren't really able to afford to run it from 1850 to 2100. So I'm wondering if that file is really needed? If you do want a transient PFT file for it, maybe you could use one that was just a smaller set of years than that one (something like 2000-2050 or something).

[ekluzek]

@jtruesdal I don't have write permission on this branch, so I made some updates to #1998 where this will actually come in for a tag. This will be combined with a few other PR's for the actual tag. This PR will auto close when the tag is made.