CTSM
CTSM copied to clipboard
ESCOMP
New CAM-MPASA grids in CTSM
10 new CAM-MPASA global grids need to be added to CTSM for development and future production runs supporting SIMA. The new grids are
mpasa480_mpasa480
mpasa120_mpasa120
mpasa60_mpasa60
mpasa30_mpasa30
mpasa15_mpasa15
mpasa12_mpasa12
mpasa15-3_mpasa15-3
mpasa7_mpasa7
mpasa5_mpasa5
mpasa3.75_mpasa3.75
Based on previous related issues and discussions with @ekluzek @billsacks the minimum level of support for each grid should include:
yr 1850 fsurdat for crop (to be used for both crop and no-crop) yr 2000 fsurdat for crop (to be used for both crop and no-crop)
Since the flanduse_timeseries files can be extremely large for global hi-res (i.e. ne120), only require these transient datasets for low resolution grids: mpasa480 mpas120
yrs 1850-2000 flanduse_timeseries for crop (to be used for both crop and no-crop) yrs 1850-2100 flanduse_timeseries for SSP5-RCP8.5 scenario, for crop (to be used for both crop and no-crop)
The surface datasets required by these grids need to have entries in namelist_defaults_ctsm.xml. Additionally, these new grids need to be added as valid values to the "res" namelist via namelist_defaults_ctsm.xml.
PE layouts and tests for the fully supported grids will be added as well. We will make sure they are all tested in the namelist testing and then do a selection of them in SMS short testing to make sure they run. We will use the comment field in the testlist xml file to note something like, "Want at least one test of mpasa120 because this grid is needed by CAM"
@MiCurry and @jtruesdal will be responsible for creating the datasets and code mods to be added via PR.
@jtruesdal do you have a table that shows the rough resolution for these grids to for example a grid size in latitude and longitude? Or a comparison to existing grid resolutions? Also are these roughly constant grid size or are they some kind of variable grid resolution? Thanks.
They are global grids with numbers representing square km. Although now that I look at it I think we also will need a mpasa240? I'll check on that.
mpasa480 480km ~4deg mpasa120 120km ~1 deg mpasa60 60km. ~0.5 deg mpasa30. 30km. ~0.25 and so on.
The higher resolution were requested by the MPAS folks.
This from @MiCurry on the 15-3 grid which is a variable resolution mesh. So all global except the 15-3. The 15-3 km mesh is a variable resolution mesh that refines from 15 km to 3 km. The 3 km portion spans roughly 60 degrees of latitude and longitude. You can see the refinement at the bottom of this page: http://mpas-dev.github.io/atmosphere/atmosphere_meshes.html.
@jtruesdal we do have a 240 km mesh: 240-km mesh (10242 horizontal grid cells). I wasn't sure to add that in the list as I wasn't sure if it was a priority, but we could definitely add it.
@ekluzek execpt the 15-3km grid, all of the other grids John has listed are quasi-uniform meshes aka roughly constant size.
@ekluzek we would like to get started making these datasets if we've identified all the appropriate requirements. Are there suggestions or changes that you would like to see? Which version of the tools should we be using to create the boundary data (latest cesm release? or a development tag?). I think we will probably run into issues with the higher resolution datasets as we generally have to make some workarounds for even quarter degree. Erik do you have any suggestions for those higher resolutions? I was also wondering if the NUOPC changes will mitigate having to create land boundary data or will we also have to work through memory and other issues on that front when dealing with these 5 and 3 km grids. Has anyone produced land data for grids that high. I would imagine that some of them may be gigantic.
jt
@jtruesdal we might need a short discussion on this. The tools are in an awkward state right now. There are plans to bring in updates that will require us to create all new surface datasets for ctsm5.2 which is coming soonish. So we've made some changes to move toward that, but are midway. So I wouldn't suggest using the latest tools. We also have plans on replacing the tool chain with a more user-friendly process. I think that is on a longer scale than what you need. But, we likely should talk for a bit about this before you start. I think we likely need to make a tag that will enable you to start your project.
@negin513 can you comment about the new tool chain here?
Anything that would allow us to move ahead at this point would be appreciated. We'd like to get this in the repo so that everything is working out of the box to allow development and validation to continue.
We do need more discussion here. But, I will say that I used ctsm5.1.dev030 to create a dataset for Louisa and Simone and I created a subdirectory beneath lnd/clm2/surfdata_map for it. So that's the version I'd recommend using for creating surface datasets right now. There's still the question about how to handle it when we go to new surface datasets with ctsm5.2 though.
Thanks Eric. Miles has started creating some of our datasets and we will make sure to use ctsm5.1.dev030 and use the new subdirectory for the surfdata. Do you know what the timeframe is for moving to the new surface datasets? Will they be in the next CESM release?
@jtruesdal we don't have a firm timeline because there are still things that need to go into ctsm5.1 series before we start the ctsm5.2 series. But, the ctsm5.1 content is pretty well defined at this point. But, yes the new datasets will certainly be in the next CESM release certainly CESM3. If there is a CESM2.3 release (which I don't think there will be) it might not make it if that were to happen soon. But I haven't heard plans for a CESM2.3 release so I'd say most likely it will be in the next CESM development release whenever that happens. Obviously it will NOT be in any older series releases like CESM2.2 or CESM2.1 though.
We talked about this as a group. We thought since we are planning on updating all of the surface datasets (in a way that's not backwards compatible) that at least getting two resolutions into the system now would be helpful. @MiCurry proposed mpasa60 and mpasa15-3 (although they likely need a new naming convention for this that includes the location of the refinement). When we go to ctsm5.2 with all new surface datasets we'll want the capability to create and run all of these other resolutions as well. @MiCurry noted that most of the above resolutions are valid options in the latest version of cime. We did wonder what the land/ocean mask it was using though. @jtruesdal suggested the tenth degree ocean would make the most sense for these high resolution cases.
@MiCurry and @jtruesdal I created a new tag I recommend you start using ctsm5.1.dev038.
Hi @ekluzek. I've created a dataset for the mpasa60_mpasa60 mesh (163842 grid cells). You can find all of the files at: /glade/scratch/mcurry/cam-mpas-mapping-domain/mpasa60. As well, @jtruesdal has copied them into the appropriate place in /glade/p/cesmdata/inputdata.
As well, I have updated a branch of @jtruesdal's fork of CTSM: https://github.com/jtruesdal/ctsm/tree/ctsm5.1.dev019_mpas_fixes. Which adds each file.
I'm not exactly sure how reviewing these files go. So let me know if there is anything I've missed or if you need something else. I'll be happy to provide them!
Hi @MiCurry I just looked at your branch and also started talking with @jtruesdal about this. It looks like to me that there are missing mapping files for mksurfdata_map to run at the new resolutions.
The branch also currently adds four new resolutions, three of which have landuse-timeseries files. From the comment above we should just add mpasa60 and mpasa15-3 for now. For ctsm5.2 we'll want to create all the resolutions.
@jtruesdal please make the branch into a PR, so we can discuss it coming in there. The branch will need to be updated to the latest, which should be easy to do.
Hi @ekluzek and @jtruesdal. Would my mpas/grids be a better branch to use for making a PR? It contains the 480, 120, 30 and 15-3. I have another branch with the 60 km, but I have not been able to successfully run a simulation on it, even after remaking the surface data file and topography file.
@ekluzek when you say mksurfdata_map mapping files do you mean the files like: map_3x3min_GLOBE map_3x3min_LandScan2004, map_5x5min_IGBP-GSDP, etc? If so, I'd be happy to add them, but I am not sure where to do that. Can you show me where to make those changes?
I think just adding the 480, 120, 60, 30 and 15-3 km should be a good goal for this PR.
The 60 km fails after the first dynamics timestep as a NaN gets into the w field. We've isolated the problem to a physics fields. We set all of the physics tendencies to 0, and the model was able to run. Currently, we are not sure if the surface data file or the topography file is causing this NaN to occur.
@MiCurry yes the map_3x3min and etc. files. They go into the map section of namelist_defaults_ctsm.xml which is near the bottom of the file. There's a set of maps for each of the supported grids.
It looks like your branch is in a newer version, but has nearly the same content. So I'll let you and @jtruesdal decide which one should become the PR to bring in.
I would keep the ability to make the 60 km grid, but yes maybe don't include the surface dataset until you get it working. But, if the problem with getting it working is on the CAM side, it doesn't matter if you include it now. You could report it as an issue in CTSM until it is working. When you say you set "the physics tendencies to 0", you mean in CAM right? If so that sounds like the issue is clearly in CAM, so it would be OK to include it in CTSM. You should also just do a CTSM I case and make sure if it works there, if so there's no reason not to include it in CTSM.
We'll need to make sure this is added both to ctsm5.1 main dev and to the ctsm5.2 alpha branch.
This already came in for ctsm5.1 and is part of the datasets made by mksurfdata_esmf in ctsm5.2. So closing.