ESCOMP
CAM CONUS regression test fails most of the time in CTSM code
Brief summary of bug
[Give a one or two sentence summary. This could be the same as the issue title if you feel that is a sufficient summary.]
CAM regression testing errors out in lnd_set_decomp_and_domain.F90 with a floating invalid error in ESMF. Occasionally the test passes.
General bug information
CTSM version you are using: [output of git describe]
alpha-ctsm5.4.CMIP7.09.ctsm5.3.068
Does this bug cause significantly incorrect results in the model's science? [Yes / No] No - the job crashes
Configurations affected: [Fill this in if known.] CONUS grid
run the test: SMS_D_Ln9_P1536x1.ne0CONUSne30x8_ne0CONUSne30x8_mt12.FCHIST.derecho_intel.cam-outfrq9s [Fill in details here.]
Important details of your setup / configuration so we can reproduce the bug
See the test results in: /glade/derecho/scratch/cacraig/aux_cam_intel_20251121170152/SMS_D_Ln9_P1536x1.ne0CONUSne30x8_ne0CONUSne30x8_mt12.FCHIST.derecho_intel.cam-outfrq9s.GC.aux_cam_intel_20251121170152/run
The first run log which failed: cesm.log.3708981.desched1.251122-161114
The rerun (second run) which passed: cesm.log.3711830.desched1.251122-162322.gz
We have baseline tests for this case that have been passing for us. And I tried the case in the exact branch tag, and it worked as well.
SMS_Ln9.ne0CONUSne30x8_ne0CONUSne30x8_mt12.IHistClm60Sp.derecho_intel.clm-clm60cam7LndTuningMode_2013Start--clm-nofireemis
Comparing the cases I see some important differences, so I'll try running a case with them. The CAM case is also with Clm50 and Cam60 so I'll try a setup that way. There's a bunch of megan and drydep settings I'll turn on as well.
