MSL 4.1.0 Reference results Modelica.Electrical.Machines.Examples.SynchronousMachines.SMPM_NoLoad

[maltelenz]

After #4031 the reference signals for Modelica.Electrical.Machines.Examples.SynchronousMachines.SMPM_NoLoad changed, so we need new reference results.

I also question why the test is not marked red in the report: https://www.ltx.de/download/MA/Compare_MSL_v4.1.0/Compare/Modelica/testrun_report.html

[beutlich]

I also question why the test is not marked red in the report: https://www.ltx.de/download/MA/Compare_MSL_v4.1.0/Compare/Modelica/testrun_report.html

To answer this question. It is marked as N/A since none of the new signal names

toPolar.y[1]
toPolar.y[2]
hallSensor.y

is available in https://github.com/modelica/MAP-LIB_ReferenceResults/blob/2be1c7e08a39d658b1326607917a0a32749e76e1/Modelica/Electrical/Machines/Examples/SynchronousMachines/SMPM_NoLoad/SMPM_NoLoad.csv

[maltelenz]

That sounds serious enough to warrant red coloring, if the reference result doesn't even contain the variables one expects to test...

[beutlich]

Well, it is neither red nor green, but inclonclusive indicating that the comparison test could not be completed with the data available. There is yellow color also in other places used, e.g., if simulation time exceeds some threshold, I guess.

[beutlich]

Well, csv-compare writes a failed marker filer indeed: compare_failed.log whereas the report marks it as UNTESTED:

[MatthiasBSchaefer]

We are only counting failed comparison. Something, that can not be compared is not a failed comparison.

We can introduce a yellow instead of a gray field for this, if this makes you happy ...

[maltelenz]

We can introduce a yellow instead of a gray field for this, if this makes you happy ...

That sounds better than gray. It is a serious issue with the test or regression result that needs solving, so it would be nice if the report can clearly mark it so it stands out.