csv-compare icon indicating copy to clipboard operation
csv-compare copied to clipboard

Set a reasonable absolute tolerance to avoid spurious negatives for close-to-zero signals

Open casella opened this issue 7 months ago • 15 comments

We had many false regressions in the MSL on signals that are supposed to be zero but aren't, due to numerical errors, see modelica/ModelicaStandardLibrary#4421

One solution is to remove those signals outright, but then you can't really make sure that they are close to zero, so that's not a good solution in general, unless those close-to-zero signals are somehow redundant.

@HansOlsson in that ticket suggests a pragmatic but actually quite effective fix for this problem: set the tube width not only based on a relative tolerance, but also on some a-priori nominal value for signals, e.g. 1e-3. This means that the error criterion should be something like:

(v - v_ref)/max(abs(v), nom) < tol

with nom = 1e-3 and tol = 0.002, the current setting for MSL testing.

@beutlich could you please take care of that at your earliest convenience?

Keeping @GallLeo in the loop.

casella avatar Jul 02 '24 11:07 casella