Add ne3np4 to list of surface dataset resolutions

[ekluzek]

For support of SCAM we need to support ne3np4 in addition to ne3np4.pg3 as pointed out by @jtruesdal.

Definition of done:

• [x] Create the needed files from a branch tag of ctsm5.3.0 (do rest on b4b-dev)
• [x] Add build of ne3np4 for same configurations as ne3np4.pg3 (1850, 2000, Hist, SSP2-4.5) to mksurfdata_esmf Makefile
• [x] Add the files into namelist_defaults
• [x] Add a test for it to the build-namelist tester
• [ ] Add a SCAM test using it to aux_clm (and ctsm_sci?)

[ekluzek]

Pinging @slevis-lmwg I put this as something to do as part of ctsm5.3.0, but we could do it in a post ctsm5.3.0 tag. It's easy to do, but involves enough steps that it could be done later.

[wwieder]

I'll let others evaluate the timing here, but I worry about scope creep and further delays for making a CTSM5.3.0 and CESM_beta03 tags

[wwieder]

Towards that end, I don't know this is show stopper for the scientific applications we need to meet for the 5.3 milestone or the "Aug" code chill, but we can discuss here or on Thurs.

[ekluzek]

I asked @jtruesdal offline and needs to have access to the files right away, but doesn't need it in a tag right away. So here's the plan

• We'll make the files with a branch tag when Derecho comes back up
• We'll add this work to b4b-dev and have it come in on the b4b-dev cycle...

[slevis-lmwg]

Steps in ctsm5.3.0 with the mksurfdata_esmf tool for generating the 1850 file (similarly for 2000). Includes correction discussed below:

./gen_mksurfdata_build 
./gen_mksurfdata_namelist --start-year 1850 --end-year 1850 --res ne3np4 --model-mesh /glade/campaign/cesm/cesmdata/inputdata/share/meshes/ne3np4_ESMFmesh_c230714_cdf5.nc --model-mesh-nx 488 --model-mesh-ny 1
./gen_mksurfdata_jobscript_single --namelist-file surfdata_ne3np4_hist_1850_78pfts_c240925.namelist --number-of-nodes 1
qsub mksurfdata_jobscript_single.sh 

For Hist and SSP2-4.5:

./gen_mksurfdata_namelist --start-year 1850 --end-year 2100 --res ne3np4 --model-mesh /glade/campaign/cesm/cesmdata/inputdata/share/meshes/ne3np4_ESMFmesh_c230714_cdf5.nc --model-mesh-nx 488 --model-mesh-ny 1 --ssp-rcp SSP2-4.5
./gen_mksurfdata_jobscript_single --namelist-file surfdata_ne3np4_SSP2-4.5_1850_78pfts_c240925.namelist --number-of-nodes 1
qsub mksurfdata_jobscript_single_Hist.sh

Jobs in the queue on derecho. Let's see if they work on the first attempt...

[jtruesdal]

Thanks so much for the instructions Sam!

jt

On Wed, Sep 25, 2024 at 5:04 PM Samuel Levis @.***> wrote:

Steps in ctsm5.3.0 with the mksurfdata_esmf tool for generating the 1850 file (similarly for 2000):

./gen_mksurfdata_build ./gen_mksurfdata_namelist --start-year 1850 --end-year 1850 --res ne3np4 --model-mesh /glade/campaign/cesm/cesmdata/inputdata/share/meshes/ne3np4_ESMFmesh_c230714_cdf5.nc --model-mesh-nx 1 --model-mesh-ny 488 ./gen_mksurfdata_jobscript_single --namelist-file surfdata_ne3np4_hist_1850_78pfts_c240925.namelist --number-of-nodes 1 qsub mksurfdata_jobscript_single.sh

For Hist and SSP2-4.5:

./gen_mksurfdata_namelist --start-year 1850 --end-year 2100 --res ne3np4 --model-mesh /glade/campaign/cesm/cesmdata/inputdata/share/meshes/ne3np4_ESMFmesh_c230714_cdf5.nc --model-mesh-nx 1 --model-mesh-ny 488 --ssp-rcp SSP2-4.5 ./gen_mksurfdata_jobscript_single --namelist-file surfdata_ne3np4_SSP2-4.5_1850_78pfts_c240925.namelist --number-of-nodes 1 qsub mksurfdata_jobscript_single_Hist.sh

Jobs in the queue on derecho. Let's see if they work on the first attempt...

— Reply to this email directly, view it on GitHub https://github.com/ESCOMP/CTSM/issues/2768#issuecomment-2375422476, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACG3NX4K5VLDVJ2UWXBUK4DZYM6RFAVCNFSM6AAAAABOMNV6WGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNZVGQZDENBXGY . You are receiving this because you were mentioned.Message ID: @.***>

[slevis-lmwg]

You're welcome. Just to be clear though, I submitted these jobs, so you don't need to repeat the work :-)

[slevis-lmwg]

@jtruesdal you will find the new files in /glade/derecho/scratch/slevis/temp_work/new_rawdata/tools/mksurfdata_esmf

surfdata_ne3np4_hist_1850_78pfts_c240925.nc
surfdata_ne3np4_hist_2000_78pfts_c240925.nc

Feel free to test.

The landuse file (for transient simulations) is not ready. It ran out of wallclock with 1 node, so I gave it a second node and resubmitted.

[slevis-lmwg]

John, the landuse file should be generated by this evening.

[slevis-lmwg]

Started testing on derecho

• Addition to the Makefile

make crop-global-SSP2-4.5-ne3
make crop-global-1850-low-res
make crop-global-present-low-res
make global-present-low-res  (unnecessary)

• ./build-namelist_test.pl
• python testing
• clm_pymods

Opening the PR next and will post updates there.

[slevis-lmwg]

@ekluzek let's discuss the last checkbox, due to my lack of familiarity with adding SCAM tests to our test-suites.

[slevis-lmwg]

From meeting with @ekluzek In testlist, change ptsRLA and ptsRLO tests to ne3np4.

[slevis-lmwg]

@jtruesdal I wonder whether you have used the above ne3np4 fsurdat file(s) successfully. If so, could you point me to a case that worked that I could look at?

[jtruesdal]

I've only tested these with CAM3d previously and they work but when I run them in SCAM it doesn't work. The format looks strange, they are using an older? lsmlon, lsmlat dimension for the variables where as everything up to this point always used gridcell for the unstructured grid formats.

[slevis-lmwg]

Interesting. I suspect that this is a shortcoming of our mksurfdata_esmf tool. What do you think @ekluzek?

[slevis-lmwg]

@ekluzek and I looked into this and eventually discovered user error on my part in a line that I listed above: ./gen_mksurfdata_namelist --start-year 1850 --end-year 1850 --res ne3np4 --model-mesh /glade/campaign/cesm/cesmdata/inputdata/share/meshes/ne3np4_ESMFmesh_c230714_cdf5.nc --model-mesh-nx 1 --model-mesh-ny 488 should have been ./gen_mksurfdata_namelist --start-year 1850 --end-year 1850 --res ne3np4 --model-mesh /glade/campaign/cesm/cesmdata/inputdata/share/meshes/ne3np4_ESMFmesh_c230714_cdf5.nc --model-mesh-nx 488 --model-mesh-ny 1

I will open an issue to alert users about this pitfall and to brainstorm how to remove the pitfall.

[jtruesdal]

Thanks. Hopefully the switch in nx/ny will allow it to recognize that it is an unstructured grid and use gridcell dimensions instead of lsmlat/lsmlon.

On Fri, Oct 25, 2024 at 1:21 PM Samuel Levis @.***> wrote:

@ekluzek https://github.com/ekluzek and I looked into this and eventually discovered user error on my part in a line that I listed above: ./gen_mksurfdata_namelist --start-year 1850 --end-year 1850 --res ne3np4 --model-mesh /glade/campaign/cesm/cesmdata/inputdata/share/meshes/ne3np4_ESMFmesh_c230714_cdf5.nc --model-mesh-nx 1 --model-mesh-ny 488 should have been ./gen_mksurfdata_namelist --start-year 1850 --end-year 1850 --res ne3np4 --model-mesh /glade/campaign/cesm/cesmdata/inputdata/share/meshes/ne3np4_ESMFmesh_c230714_cdf5.nc --model-mesh-nx 488 --model-mesh-ny 1

— Reply to this email directly, view it on GitHub https://github.com/ESCOMP/CTSM/issues/2768#issuecomment-2438633542, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACG3NX22NKWN2J4HWXWVXZLZ5KK5JAVCNFSM6AAAAABOMNV6WGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZYGYZTGNJUGI . You are receiving this because you were mentioned.Message ID: @.***>

[slevis-lmwg]

@jtruesdal I generated new files. I gave them identical names as before:

-rw-r--r-- 1 slevis cseg 229712212 Oct 28 13:35 ../../lnd/clm2/surfdata_esmf/ctsm5.3.0/landuse.timeseries_ne3np4_SSP2-4.5_1850-2100_78pfts_c240926.nc
-rw-r--r-- 1 slevis cseg  17072072 Oct 28 13:37 ../../lnd/clm2/surfdata_esmf/ctsm5.3.0/surfdata_ne3np4_hist_1850_78pfts_c240925.nc
-rw-r--r-- 1 slevis cseg  17072072 Oct 28 13:40 ../../lnd/clm2/surfdata_esmf/ctsm5.3.0/surfdata_ne3np4_hist_2000_78pfts_c240925.nc

Our CTSM SCAM test SMS_D_Ld1_Mmpi-serial.ne3_ne3_mg37.I2000Clm50SpRs.derecho_intel.clm-ptsRLA fails with a new error, so I wanted to ask you this: @ekluzek and I were comparing the two ne3 mesh files, and Erik found it strange that the np4 file has maxNodePElement = 5. Do you have thoughts about that?

1c1
< netcdf ne3np4_ESMFmesh_c230714_cdf5 {
---
> netcdf ne3pg3_ESMFmesh_c221214_cdf5 {
3,5c3,5
<       nodeCount = 510 ;
<       elementCount = 488 ;
<       maxNodePElement = 5 ;
---
>       nodeCount = 488 ;
>       elementCount = 486 ;
>       maxNodePElement = 4 ;

[slevis-lmwg]

UPDATE

I looked at the equivalent ne30 mesh files, and they differ in maxNodePElement in the same way, so probably the maxNodePElement = 5 is fine.

I submitted the same ptsRLA test with ne30_g17, ne30pg3_t232, and ne3pg3_ne3pg3_mg37 and they all failed, while with f45_f45_mg37 the test works.

[slevis-lmwg]

...the error seems to always point to urban code.

[jtruesdal]

I’m testing the single column version of ne3np4 and they seem to be working after a few modifications to the scam portion of a few clm files. Could you add my fixes as part of this PR? I’m assuming you need these mods for your failing se tests. I have one more BFB test to make before signing off on the mods.

On Tue, Oct 29, 2024 at 12:55 PM Samuel Levis @.***> wrote:

...the error seems to always point to urban code.

— Reply to this email directly, view it on GitHub https://github.com/ESCOMP/CTSM/issues/2768#issuecomment-2445089802, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACG3NX5WKYZGNIES5FNIKVDZ57K23AVCNFSM6AAAAABOMNV6WGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINBVGA4DSOBQGI . You are receiving this because you were mentioned.Message ID: @.***>

[slevis-lmwg]

Thank you, @jtruesdal. Let me know where to get the mods from. Unless you prefer to push them to my PR (#2835) yourself. If so, let me know, and I will give you appropriate permissions.

[jtruesdal]

I'm hoping to get to the BFB test today and will also run the CAM regression suite to make sure I haven't broken anything. I'll hand off the mod when that is complete (Thur or Fri). Can I make a PR against the branch listed in #2835 and then you could just merge them in?

[slevis-lmwg]

Yes, that'll work.

[jtruesdal]

I made a PR to your branch with the additional mods to get SCAM running. Could you try them out and see if they work for you.

[ekluzek]

@slevis-lmwg is there anything for me to do here -- other than review? If not I'll remove myself as assignee.

[slevis-lmwg]

I made a PR to your branch with the additional mods to get SCAM running. Could you try them out and see if they work for you.

Thank you, @jtruesdal. I merged your branch to mine and pushed to the PR. I will test when derecho comes back.

[slevis-lmwg]

I'm marking my "sprint" card DONE here and opening a new card in the PR #2835 for the remaining work.

I'm not marking the issue itself DONE until I merge the PR.

[ekluzek]

This will close when the cleanup branch comes in #2950