ESCOMP
Turn `use_bedrock` on for FATES by default
Description of changes
Updates the fates use_bedrock defaults in the namelist defaults to be .true. by default.
Specific notes
See https://github.com/ESCOMP/CTSM/issues/1888#issuecomment-1312023979 for the initial confirmation that the model works as expected.
Contributors other than yourself, if any: @KatieMurenbeeld @jkshuman
CTSM Issues Fixed (include github issue #): #1888
Are answers expected to change (and if so in what way)? Yes
Any User Interface Changes (namelist or namelist defaults changes)?
Testing performed, if any: (List what testing you did to show your changes worked as expected) (This can be manual testing or running of the different test suites) (Documentation on system testing is here: https://github.com/ESCOMP/ctsm/wiki/System-Testing-Guide) (aux_clm on cheyenne for intel/gnu and izumi for intel/gnu/nag/pgi is the standard for tags on master)
NOTE: Be sure to check your coding style against the standard (https://github.com/ESCOMP/ctsm/wiki/CTSM-coding-guidelines) and review the list of common problems to watch out for (https://github.com/ESCOMP/CTSM/wiki/List-of-common-problems).
Running the fates regression tests on this is resulting in RUN failures with a floating point exception. The truncated stack trace is below. Folder of tests is here: /glade/u/home/glemieux/scratch/ctsm-tests/tests_bedrock-fates-retest
17 0:MPT ERROR: Rank 0(g:0) received signal SIGFPE(8).
...
147 0:MPT: #6 soilwaterplantsinkmod::compute_effecrootfrac_and_verttransink_default (
148 0:MPT: bounds=..., num_filterc=1, filterc=..., soilstate_inst=...,
149 0:MPT: waterfluxbulk_inst=...)
150 0:MPT: at /glade/u/home/glemieux/ctsm/src/biogeophys/SoilWaterPlantSinkMod.F90:409
...
I highlighted the SoilWaterPlantSinkMod lines as this is the first point in the trace that is specific to a ctsm line of code (and not a dependency). The relevant lines are here:
https://github.com/ESCOMP/CTSM/blob/979c71f70ad40cb4ff3f6d528d8b6016a8e5f028/src/biogeophys/SoilWaterPlantSinkMod.F90#L402-L414
Printing out the variables on that line during runtime shows that rootr_patch is NaN for some patch indices, so this likely is some sort of initialization issue.
Check to see if https://github.com/NGEET/fates/issues/732 is addressed by this.
@glemieux what is the status of this one?
Extremely stale. It's pretty far down the queue at the moment and hasn't been tested with recent tags. Do you want to close it out for now?
I'll leave it open as a draft. The changes are so small that they will easily port to the latest version. If they were involved it might be good to close and reopen later. This probably just needs to be made scientific priority to get it working and tested.
@glemieux I'm wondering if this is still something we should do? And should it be done for the release or afterwards? Making FATES run with the same bedrock layer as for non-FATES does seem desirable to me. But, it changes answers and needs scientific validation.
@glemieux I'm wondering if this is still something we should do? And should it be done for the release or afterwards? Making FATES run with the same bedrock layer as for non-FATES does seem desirable to me. But, it changes answers and needs scientific validation.
@ekluzek is the freeze March 28? We might be able to prioritize is. @rgknox what do you think? I've merged in ctsm5.3.021 and retested to validate its still failing in the same manner. My guess is the issue is that the patch%wtcol is NaN.
Bringing this up to ctsm5.3.041 and spot testing with ERP_Ld9.f45_f45_mg37.I2000Clm50FatesCruRsGs.derecho_intel.clm-FatesCold I'm seeing a carbon balance error now:
1 dec1678.hsn.de.hpc.ucar.edu 382: cbalance warning at c = 3 -4.909853808133992E-006
2 dec1678.hsn.de.hpc.ucar.edu 382: 912.000015072109
3 dec1678.hsn.de.hpc.ucar.edu 382: column cbalance error = -4.909853808133992E-006 3
4 dec1678.hsn.de.hpc.ucar.edu 382: is fates column? = T
5 dec1678.hsn.de.hpc.ucar.edu 382: Latdeg,Londeg= 66.0000000000000 165.000000000000
6 dec1678.hsn.de.hpc.ucar.edu 382: begcb = 912.000010048073
7 dec1678.hsn.de.hpc.ucar.edu 382: endcb = 912.000015072109
8 dec1678.hsn.de.hpc.ucar.edu 382: delta store = 5.024036454415182E-006
9 dec1678.hsn.de.hpc.ucar.edu 382: --- Inputs ---
10 dec1678.hsn.de.hpc.ucar.edu 382: fates litter_flux = 1.141826462811892E-007
11 dec1678.hsn.de.hpc.ucar.edu 382: --- Outputs ---
12 dec1678.hsn.de.hpc.ucar.edu 382: hr = 0.000000000000000E+000
13 dec1678.hsn.de.hpc.ucar.edu 382: -1*som_c_leached = 3.751963665376138E-024
14 dec1678.hsn.de.hpc.ucar.edu 382: iam = 254: local column index = 3
15 dec1678.hsn.de.hpc.ucar.edu 382: iam = 254: global column index = 7112
16 dec1678.hsn.de.hpc.ucar.edu 382: iam = 254: global landunit index = 2735
17 dec1678.hsn.de.hpc.ucar.edu 382: iam = 254: global gridcell index = 1279
18 dec1678.hsn.de.hpc.ucar.edu 382: iam = 254: gridcell longitude = 165.0000000
19 dec1678.hsn.de.hpc.ucar.edu 382: iam = 254: gridcell latitude = 66.0000000
20 dec1678.hsn.de.hpc.ucar.edu 382: iam = 254: column type = 1
21 dec1678.hsn.de.hpc.ucar.edu 382: iam = 254: landunit type = 1
22 dec1678.hsn.de.hpc.ucar.edu 382: ENDRUN:
23 dec1678.hsn.de.hpc.ucar.edu 382: ERROR in CNBalanceCheckMod.F90 at line 384
24 dec1678.hsn.de.hpc.ucar.edu 382: Image PC Routine Line Source
25 dec1678.hsn.de.hpc.ucar.edu 382: cesm.exe 000000000101D587 shr_abort_mod_mp_ 106 shr_abort_mod.F90
26 dec1678.hsn.de.hpc.ucar.edu 382: cesm.exe 00000000005D8B9F abortutils_mp_end 98 abortutils.F90
27 dec1678.hsn.de.hpc.ucar.edu 382: cesm.exe 0000000000E66B32 cnbalancecheckmod 384 CNBalanceCheckMod.F90
28 dec1678.hsn.de.hpc.ucar.edu 382: cesm.exe 00000000008C3F87 cnvegetationfacad 1229 CNVegetationFacade.F90
29 dec1678.hsn.de.hpc.ucar.edu 382: cesm.exe 00000000005E86FD clm_driver_mp_clm 1177 clm_driver.F90
Results: /glade/u/home/glemieux/scratch/ctsm-tests/tests_pr1902-bedrockchk
@glemieux I put this as a post CESM3.0 activity, since we can't change answers for FATES-SP. And marked it as low priority.