CMEPS icon indicating copy to clipboard operation
CMEPS copied to clipboard

Write mapping files that can be reused on restart

Open billsacks opened this issue 3 years ago • 3 comments

From a discussion with @mvertens, @jedwards4b and @gustavo-marques today: We want to consider adding the ability to write files containing the initialization-generated mapping weights and then read these files back in on restart rather than redoing ESMF_FieldRegridStore each time the model is restarted.

A few specific notes on this:

  1. We should do some timings to verify that it's actually generally faster to read already-generated mapping files than just regenerating them. (It seems likely to be faster, but it's possible that the I/O is actually slower than the regrid weight generation.)
  2. The dumped regridding weights depend on processor layout / decomposition. @gustavo-marques 's thought is that, in the same place where we currently detect a change in processor layout and issue a message about needing to rebuild, we could also remove any of these generated mapping files.
  3. For CESM, we should put these files in the recently-added subdirectory of the run directory named init_generated_files. This has the advantages of (a) This directory is automatically removed when you rerun a system test, to ensure that the system test is running the same way each time, unless you are running the specific new test type REUSEINITFILES that explicitly tests rerunning with the reuse of these init files (and ensures that your run is bit-for-bit when doing so); (b) We could have CIME remove this entire directory when changing the PE layout... this may be overkill (e.g., it removes some CTSM-generated initialization files that don't actually need to be removed when changing the PE layout), but that's probably okay to keep things simple and robust.

billsacks avatar Jan 06 '23 19:01 billsacks

I'll just add a note that this kind of thing might be good to do for debugging so you can visualize what is happening with the remapping. It might be easier to figure out the mapping based on looking at mapping files rather than querying ESMF remapping objects while the code is running.

ekluzek avatar Mar 22 '23 21:03 ekluzek