models that reject "obs" in model interpolate for being two high or two low - problem for column obs_defs

[hkershaw-brown]

There are several model_mods that have model_interpolate calls that will fail for locations above or below a given location. For forward operators where a whole column, or a surface value is needed this causes the forward operator to fail. RTTOV forward operators are where people are hitting this but I believe it applies to any forward operator (obs_def) that requires a full column or some surface value of a qty.

model_interpolate does not get the observation as input. A call to model_interpolate requests the value of a quantity at a location. There are several (3-4 noted below but maybe more) where the model_mod::model_interpolate is treating model_interpolate as a way to reject observations, rather than simply providing the value of the quantity at the location.

• mpas_atm https://github.com/NCAR/DART/blob/464aa57a1261fa098e5f0f999bad41a464f8d7db/models/mpas_atm/model_mod.f90#L1268-L1270

• wrf https://github.com/NCAR/DART/issues/366 rttov obs_def is using model_interpolate to count the levels so one-off.

• cam se/fv discarding_high_obs so radiance fwd operators will fail (rttov needs whole column) https://github.com/NCAR/DART/issues/438

[kdraeder]

This was discussed extensively in #296 for cam-fv. One (partial?) fix was to move the discarding_high_obs test from model_interpolate to get_close_{obs,state}. But that apparently was not merged into main. That issue morphed into the problems with incorrect QC values in obs_seq.final, but much of the discussion seems relevant to this issue.

[hkershaw-brown]

thanks Kevin, that discussion also continues in https://github.com/NCAR/DART/discussions/436.