E3SM-Project
Incompatible domain grid coordinate with bi-grid ne1024pg2
A run using grid ne1024pg2_oRRS19to6v3 (available in branch wlin/atm/ne1024pg2_oRRS18to6v3, PR #1131 failed with
ERROR: (seq_domain_check_grid) incompatible domain grid coordinates
12273: seq_domain_check_grid - n:482 d1: 231.582371 d2: 231.582371 diff: 0.00000000000142 eps: 0.00000000000100
The grids seen by atm and lnd domains have differences from 12th decimal point in lat/lon. Below is backtracing of the code flow from where it was aborted.
shr_abort_mod_mp_ 114 shr_abort_mod.F90
seq_domain_mct_mp 693 seq_domain_mct.F90
seq_domain_mct_mp 378 seq_domain_mct.F90
cime_comp_mod_mp_ 1989 cime_comp_mod.F90
The differences arise from the two calls in the following if-block
if (atm_present .and. lnd_present .and. samegrid_al) then
if (iamroot) write(logunit,F00) ' --- checking atm/land domains ---'
call seq_domain_check_grid(atmdom_a%data, lnddom_a%data, 'lat' , eps=eps_axgrid, mpicom=mpicom_cplid, mask=maskl)
call seq_domain_check_grid(atmdom_a%data, lnddom_a%data, 'lon' , eps=eps_axgrid, mpicom=mpicom_cplid, mask=maskl)
endif
It is likely because different versions of grid and/or map files are used during runtime or for generating certain input files. It would be helpful if we know which files are involved to feed the domain/grid data to the above atmdom_a%data and lnddom_a%data.
@wlin7 - you could try changing the tolerance in env_run, since it is so close to passing. I think that setting is:
<entry id="EPS_AGRID" value="1.0e-12">
<type>real</type>
<desc>Error tolerance for differences in atm/land lat/lon in domain checking</desc>
</entry>
and it looks like you just need to bump it up slightly
Thanks, @jonbob . Useful to relax the tolerance to see how far this first test with the grid can go. The diff shown above is just an example of one cell. The max difference can be much larger. When EPS_AGRID=1e-12, it captured diff as large as 9e-12. When increase EPS to 2e-11, it captured as large as 8e-11. Tentatively bump it to a 1e-7, no larger difference is detected and the model can run. Further tests to scale it back after the run with 1e-7 is completed.
I've ran into this before, I think the cause was using a different algorithm in one of the mapping files that was used to generate the domain files. I believe it's the atmosphere to ocean or ocean to atmosphere flux maps that are relevant here, so you might double check that the versions of those you're specifying in config_grids.xml are identical to those used when generating the domain files.
