Deciding on a accepted tolerance for the fieldcompare

[valentin-seitz]

To give a bit of perspective: When running the systemtests we execute the coupled participants in the given configuration and then compare exported coupling meshes to a known "good solution" for that participant combination.

Since we compile with a lot of potential hardware specific optimizations turned on, having different CPU capabilities might lead to floating point differences in the execution.

Now to stop the systemtests from failing i just guesstimated a number out of thin air which was a relative tolerance of 3e-7.

I think is a rather high value, but i also dont have any better estimation.

This issue is mainly here to discuss how to tackle the problem.

[MakisH]

What I remember from our discussion:

Even when running the exact same tests inside the Docker containers, but on different systems (laptop and GitHub runner), floating-point results would have some minor differences, which are currently not fully explainable.

We suspect the following factors:

• Different underlying Linux kernel
• Different CPUs and optimization for different CPUs

[uekerman]

Different CPUs and optimization for different CPUs

This can indeed lead to floating-point differences.

But I don't understand the title here, RelTol should be the solution, no?

[valentin-seitz]

I added a more verbose description of the issue

[uekerman]

Do the tests run with 3e-7?

I guess, we should be able to go smaller, but will be try-and-error. Everything we do is double precision, so sth like 1e-13 should be achievable?