lapack icon indicating copy to clipboard operation
lapack copied to clipboard

Tests always have zero exit status

Open christoph-conrads opened this issue 3 years ago • 0 comments

The Makefile-based build runs all tests and then shows a test summary. In CMake, all tests are run, the test summary script is called but with the default CMake settings, the testsuite will never fail and never show the test summary because

  • failing tests are detected by non-zero exit status and
  • test output is suppressed by default.

As far as I can see, all tests always exit with a zero status even if there was a problem. This breaks testing and this behavior makes it impossible to do any sane scripting on UNIX (think of distribution packers that run make test && make install; this will always succeed). This behavior makes it also impossible to tell apart critical test failures (e.g., not recognizing negative matrix dimensions) from spurious test failures (e.g., convergence failures for numerically hard test problems).

christoph-conrads avatar Feb 24 '21 20:02 christoph-conrads