Add a test for two-way irrigation

[evasinha-pnnl]

I am using recent master for running crops with two way irrigation on but my run is failing with error ERROR in BalanceCheckMod.F90 at line 740 . I did try to debug this error using the debug option but was not making a lot of progress since the with debug ON the model crashes at an earlier NaN value.

When I ran the same case using an older branch I was able to get past the error in BalanceCheckMod.F90 and the model was running fine, which means that some recent changes in master may be causing the two way irrigation run to crash. I think there are no tests for two-way irrigation in the master and maybe adding it will identify the cause of the error as well as prevent future code breaks.

My directories on chrysalis: Run - /lcrc/group/e3sm/ac.eva.sinha/E3SMv2/20231219_CBGCv2.r05.chrysalis.I1850GSWCNPWFMCROP.adsp/run Case - /lcrc/group/e3sm/ac.eva.sinha/E3SMv2/20231219_CBGCv2.r05.chrysalis.I1850GSWCNPWFMCROP.adsp/case_scripts

[evasinha-pnnl]

@hydrotian I wanted to check if you have any insights on the error I have described above and/or the inclusion of test for two way irrigation.

[hydrotian]

@evasinha It's a bit hard to track down the changes as Xiaoying's branch was branched out 2 years ago. In fact, I've implemented a two-way irrigation test before (https://github.com/E3SM-Project/E3SM/pull/4676) but I don't know why the tests got passed in the master PRs (maybe the test was too short?). I'm not sure if it makes sense to test your configuration with master at different time to narrow down when the change could possibly happen. Maybe start with the master where Xiaoying branched out https://github.com/E3SM-Project/E3SM/commit/5a230dde4512895c7c1440113e7b2e724966da1e (is it possible that your config worked because of the developments happened on Xiaoying's branch?).

[evasinha-pnnl]

Thanks @hydrotian

I submitted SMS_Ly2.r05_r05.IELM.chrysalis_intel.elm-V2_ELM_MOSART and ERS.r05_r05.IELM.chrysalis_intel.elm-V2_ELM_MOSART_features (2 year teast) tests on chrysalis and both completed successfully.

[evasinha-pnnl]

The misleading soil balance error is likely being caused by extremely large qflx_surf_irrig (surface water irrigation flux).

Extremely small pgwgt (pft's weight in gridcell) causes qflx_supply (irrigation supply) to be very large:

qflx_supply(p) = atm2lnd_vars%supply_grc(g)*(1-wt_lunit(g,tpu_ind,istdlak))/pgwgt(p)

that causes qflx_surf_irrig to be very large

qflx_surf_irrig(p) = qflx_supply(p)/irrigated_ppg(g)

So, in my simulation the pgwgt for irrigated soybean cft was 1.163E-012 and that resulted in qflx_surf_irrig to be 2477678.32 (mm H2O /s).

I guess we need to have some kind of check if pft weights less than a threshold convert to zero?

@hydrotian @bishtgautam @qzhu-lbl do you have others suggestions for resolving this problem.

[hydrotian]

@evasinha Thanks for finding the cause! I would suggest to set a user-defined minimum fraction of pft in the namelist file. Any pft that's smaller than this threshold value will be set to zero, and the remaining fraction will be added to the largest pft in this gridcell to make sure everything adds to 100.

[evasinha-pnnl]

@hydrotian I thought more about the issue and also checked the mksurfdat code and realized that such small fractions should not exist in our surface dataset or landuse timeseries. So, I wrote a script to check the surface dataset for the occurrence of all small fractions and updated them based on the method used in remove_small_cover in E3SM/components/elm/tools/mksurfdata_map/src/mkutilsMod.F90.

I then re started a new ad-spinup with the revised surface dataset. It has been going on for few years now and has not yet crashed.