cppfront icon indicating copy to clipboard operation
cppfront copied to clipboard

regression-tests: run tests using bash

Open hugolm84 opened this issue 2 years ago • 2 comments

Convenience script for running regression tests on Linux. Current results on Ubuntu and gcc11:

./run-tests.sh
TRANSPILE:
34 SUCCESS
8 FAILED
./test-results/mixed-lifetime-safety-pointer-init-3.cpp2
./test-results/mixed-initialization-safety-1.cpp2
./test-results/mixed-lifetime-safety-pointer-init-2.cpp2
./test-results/mixed-initialization-safety-2.cpp2
./test-results/mixed-lifetime-safety-pointer-init-1.cpp2
./test-results/pure2-bounds-safety-pointer-arithmetic-error.cpp2
./test-results/pure2-lifetime-safety-pointer-init-1.cpp2
./test-results/pure2-lifetime-safety-reject-null.cpp2
COMPILE:
33 SUCCESS
1 FAILED
./test-results/mixed-captures-in-expressions-and-postconditions.cpp
./run-tests.sh: line 26: 184219 Segmentation fault      (core dumped) ./$filename >> ./$filename-output 2>&1
./run-tests.sh: line 26: 184231 Aborted                 (core dumped) ./$filename >> ./$filename-output 2>&1
RUN:
31 SUCCESS
2 FAILED
./test-results/mixed-lifetime-safety-and-null-contracts.cpp.out
./test-results/mixed-bounds-safety-with-assert.cpp.out

hugolm84 avatar Sep 26 '22 13:09 hugolm84

Thanks. I'm slowly overhauling the regressing testing to run the tests, and I plan to also do that in bach. I'll defer this until then, I won't get to it immediately but perhaps after the Kona meeting. Again, thanks!

hsutter avatar Oct 01 '22 23:10 hsutter

I was actually just beginning to work on a very similar PR, glad I saw this one before I got too into it. I would definitely propose merging this now, and potentially reverting it later when the regression tests are overhauled.

PastaPastaPasta avatar Oct 21 '22 18:10 PastaPastaPasta

I've now finished adding support for actually running the regression tests using MSVC, GCC, and Clang. Please see the new stuff under /regression-tests including checked-in results of executing the tests.

I know there's more that could be done, including CI, but for now I'm happy with just running four batch/shell scripts by hand (one to put all the Cpp2 tests through cppfront, and then one each per compiler to build and run the Cpp1 output).

Thanks!

hsutter avatar Dec 13 '22 01:12 hsutter