Create "super sparse" grid to investigate a new spinup approach for CTSM

[wwieder]

I've been in discussions with with Samar and Andrew (@samarkhatiwala & @ awil5504) about trying an Anderson, or sequence, acceleration for land model spinup (see this this paper. Ideally this method could be adapted for MIMICS and FATES cases, but for now we're starting with default CLM configurations (BCG-Century)

Preliminary work that Andrew has done with a single point NEON case looks promising, but now it would be helpful to create a super sparse grid to get a more complete picture of what restart files will look like. This still could be just a few points, but that selects grid cells with multiple pfts and land units (including crops). Ideally it could include an arctic, temperate and tropical grid cell.

How can we go about creating a surface dataset and mesh file for a file like this? @djk2120 and @linniahawkins how is the sparse grid being created for the PPE? Is this something we can do relatively easily?

Note, if this project moves forward, we can start a larger project board for other features associated with the sequence acceleration.

[djk2120]

Linnia and I use the sparsegrid a lot, but neither of us has actually constructed one :) Keith did it for our simulations (all mct) Adrianna has done it for some nuopc simulations.

I think it is relatively easy to do, but there are some land mines you have to avoid, so I would talk to Adrianna

[wwieder]

@adrifoster can you help us create a mesh file, once we pick a few point to test out?

[adrifoster]

Sure, send me the lat/lons and I can make it for you.

[adrifoster]

Sure, send me the lat/lons and I can make it for you.

well actually I can just make a script with an input set of lat/lons. What format do you think they will be in? just a list?

[wwieder]

Once I can look at a surface dataset again I'll just list them here if that's OK? @linniahawkins also mentioned she could pick a few of her favorite points from the sparse grid (although we also want crops in for this application)

[linniahawkins]

Oh, if you need crops on I don't think it will work to select cells from the PPE sparse grid.

[wwieder]

Specifically from a CTSM5.2 surface dataset (/glade/campaign/cesm/cesmdata/inputdata/lnd/clm2/surfdata_esmf/ctsm5.2.0/surfdata_0.9x1.25_hist_1850_78pfts_c240216.nc)

I thing the following points should be adequate for starters.

lsmlat, lsmlon = 77, 251 [tropical w/ crop] lsmlat, lsmlon = 139, 217 [temperate w/ crop] lsmlat, lsmlon = 164, 176 [arctic/boreal]

[wwieder]

@adrifoster provided this! Thanks. I'm marking this as done

[slevis-lmwg]

This issue can remain closed, but I wanted to post an update regarding use of the 400-point sparse grid in the same work.

@awil5504 pointed out in a recent meeting that the PE layout proposed here worked fine for Century but was order 3 to 5x slower for Mimics. For the purposes of @awil5504's work, we are concerned with speed and not efficiency, so I have tried doubling and quadrupling the number of tasks with success. The following output appears in the med.log files:

• @awil5504's simulation: # simulated years / cmp-day = 49.623
• @slevis-lmwg's clone: # simulated years / cmp-day = 48.029 SANITY CHECK
• PIO_STRIDE=32: # simulated years / cmp-day = 48.087 NO BENEFIT but kept for subsequent tests
• 2x NTASKS: # simulated years / cmp-day = 85.259
• 4x NTASKS: # simulated years / cmp-day = 145.552
• 6x NTASKS: # simulated years / cmp-day = 203.426 LAST MIMICS TEST
• 6x NTASKS: # simulated years / cmp-day = 209.087 FIRST CENTURY TEST
• 1x NTASKS: # simulated years / cmp-day = 52.110

All the corresponding env_mach_pes.xml and env_run.xml are in this case: /glade/u/home/slevis/cases_mimics/400_pt_sparse

@awil5504 your much larger MIMICS v. CENTURY performance difference may relate to changes in history output or something along those lines.

[slevis-lmwg]

Given the total of 400 grid cells, you can try even 6x NTASKS for some additional benefit if you wish.

[wwieder]

Thanks for looking at this, Sam. Out of curiosity, what throughput do the century runs get with the sparse grid?

[awil5504]

@wwieder I was tending to see somewhere between 280-310 years/day with the same PE layout. Haven't tried century with Sam's new layout

[slevis-lmwg]

I will post an update with 6x NTASKS and then with Century in a little bit.

[slevis-lmwg]

Updated my performance table above.

[wwieder]

Thanks Sam. I'm surprised that @awil5504 was reporting ~300 year per day, but from what you're showing MIMICS and CENTURY are pretty comparable (which makes me feel better).