Feature request: MOM6 assimilate scripting for CESM3

[hkershaw-brown]

Use case

Scripting (assimilate) for MOM6 for cesm

Is your feature request related to a problem?

People want to use CESM3 beta (MOM6), so not a problem, just users.

Describe your preferred solution

Here is the scripting we have been using for CROCODILE (regional MOM6, but currently scripting is the same for DART) https://github.com/CROCODILE-CESM/DART/blob/main/models/MOM6/cesm_scripts/assimilate.sh This was to test cycling for use in Crocodile notebooks. Still need to add inflation.

This is the routine in cime that calls the assimilate script: https://github.com/ESMCI/cime/blob/d0b22795c9c3d2869d63964544f26e5e088a05bc/CIME/case/case_run.py#L420-L434

Note, since mostly we have been working with regional MOM6 which is under development, the filenames have been changing (see Enricos commits), so the output from MOM6 may not be in its final form.

Google doc notes on setting up mom6 cases - not I am not sure of the current supported CESM tags. https://docs.google.com/document/d/1CGkvM_0CXy3b7gE2xiTEN2IcrTI9WDTGBR1UghIUIm4/edit?tab=t.0

Describe any alternatives you have considered

List any alternative solutions or workarounds you have considered.

• the run_or_sub routine will try to import python first: https://github.com/ESMCI/cime/blob/d0b22795c9c3d2869d63964544f26e5e088a05bc/CIME/utils.py#L680 Currently just have a shell script. Using python would let you import the CIME stuff.

• There is a PRERUN and POSTRUN script if you look at the cime workflows documentation: https://esmci.github.io/cime/versions/master/html/users_guide/workflows.html which may be better for some setup/teardown stuff, but it actually looks like pre and post run are applicable to E3SM only: https://github.com/ESMCI/cime/blob/d0b22795c9c3d2869d63964544f26e5e088a05bc/CIME/case/case_run.py#L450-L469

[hkershaw-brown]

Note: MOM6 multi instance user reported problems with MOM6 only using one of the restart files.

[kdraeder]

Kate Boden is (also?) reporting a problem like that when trying to use a RUN_TYPE=hybrid case. I've found a MOM6 bug and a user error. MOM6: buildnml looks for a single instance restart file (no instance number) and fails if it is not found. In multi-instance cases it should look for files with an instance number. User: needs to provide non-default restart file names in user_nl_mom_#### in the CASEROOT. The defaults have no instance number. This can be viewed as another MOM bug.

[hkershaw-brown]

which tag are you/Kate using Kevin?

[kdraeder]

which tag are you/Kate using Kevin?

cesm3_0_beta06

[hkershaw-brown]

https://github.com/ESCOMP/MOM_interface/issues/275

[hkershaw-brown]

https://bb.cgd.ucar.edu/cesm/threads/optimize-ensemble-configuration-with-nuopc-multi_driver-true.9912/

[hkershaw-brown]

cesm3_0_beta06 RUN_TYPE=startup 3 member ../CESM/cime/scripts/create_newcase --res TL319_t232 --compset G_JRA --case g.e22.G_JRA.TL319_t232.001.multi.3 --project p86850054 --multi-driver --ninst 3

Not seeing problems, 3 member case. ran 5 days, +0.25C ens member 2, +0.12C ens member 3 to make the ensemble members different. ran 5 days 3 different restart files

[kdraeder]

Thanks for testing! Does your test also work with RUN_TYPE = hybrid? Is your rundir populated with a MOM restart file that has no instance number in the name? If it is, then MOM will happily use that, even though it is not the right file(s) for multi-instance.

I dug up some relevant code for Kate, which shows why 'hybrid' leads to incorrect file names (no instance numbers): The default values come from param_templates/MOM_input.yaml

        value:
            $RUN_TYPE == "hybrid":
                = f'./{$RUN_REFCASE}.mom6.r.{$RUN_REFDATE}-{$RUN_REFTOD}.nc'

or param_templates/json/MOM_input.json

      "THICKNESS_FILE": {
         "description": "The name of the thickness file",
         "datatype": "string",
         "value": {
            "$RUN_TYPE == \"hybrid\"": "= f'./{$RUN_REFCASE}.mom6.r.{$RUN_REFDATE}-{$RUN_REFTOD}.nc'"

There should be a place where these are modified or replaced by names with instance numbers, but I haven't found it yet.

[hkershaw-brown]

Does your test also work with RUN_TYPE = hybrid

I have not tested this yet. I am just seeing if I can reproduce the problem with RUN_TYPE=startup. Which I cannot, so this is why we haven't seen the problem in Crocodile.

Is your rundir populated with a MOM restart file that has no instance number in the name?

No.

[kdraeder]

Kate fixed the hybrid default file name problem by specifying the names explicitly in the user_nl_mom_#### files: THICKNESS_FILE, TS_FILE, VELOCITY_FILE, AGE_IC_FILE.