modelica
MSL 4.1.0 Regressions - Math
The following models fail in result comparison. Tested revision: f9bddf86 (2024-02-16)
No signals to compare, define more comparison signals:
-
[x] ModelicaTest.Math.TestMatricesExamplesSolveLinearEquations - Updated comparisonSignals.txt? - Updated reference files? No change needed I say
-
[x] ModelicaTest.Math.TestNonlinear - Updated comparisonSignals.txt? - Updated reference files? No change needed I say
-
[x] Release notes check: Are all classes mentioned which could lead to result changes in user models?
Useful Links
Current comparison report: https://www.ltx.de/download/MA/Compare_MSL_v4.1.0/comparison_report_overview.html -> Reference result test -> Comparison
Comparison signal definitions: https://github.com/modelica/ModelicaStandardLibrary/tree/master/Modelica/Resources/Reference/Modelica https://github.com/modelica/ModelicaStandardLibrary/tree/master/ModelicaTest/Resources/Reference/ModelicaTest
Reference results: https://github.com/modelica/MAP-LIB_ReferenceResults
The tested functionality works, I don't think we need to define comparison signals for these tests.
What they do is to test functions (by calling other functions). Having outputs from the Example-functions (that already exists in Modelica.Math - if they were in ModelicaTest it would be different) so that we can store them in the test-cases to have comparison signals seems too impactful for just a test - and may also lead people to use the examples as useful functions, which we don't want.
What I would propose is that:
- We update the example-functions in Modelica.Math to have some asserts (with fairly large allowed error) to test that the functions actually work. In most cases the "diff" is already printed, we just need to assert that it is small.
- For Modelica.Math.Nonlinear.Examples.solveNonlinearEquations2 assert that the forward error (residual) is small. Currently it just prints the numerical solution, and just inserting that value in the functions give zero or practically zero.
- For Modelica.Math.Nonlinear.Examples.quadratureLobatto2 compute the analytic solutions and compare with that:
- The first integral is analytically [-cos(x)] with bounds a1 and b1 giving 0.45969769413186023
- The second integral is analytically [-cos(x*w)/w] with bounds a2 and b2 giving 0.3124907702476344
- The third integral is [F(u; k^2)] (incomplete elliptic integral of the first kind) with bounds a3 and b3 (and with given values it simplifies to K(1/2) for the complete elliptic integral) and that should give 1.8540746773013719184338503471952600462175988235217669055859280450... - https://en.wikipedia.org/wiki/Elliptic_integral
This is not urgent for the release.
Looking more I realize that 2-variants are just generalizations of 1-variants (with inputs replacing fixed values); so there's less need to modify them - as we already test the functionality. It would also be harder to make good tests for 2-variants - as people may try weird inputs.
I would thus propose to close this. Merging #4344 for 4.1.0 release avoids having to manually check those cases.
