blt
blt copied to clipboard
LLNL
Ordering or depending tests
Just using the calc_pi example...
[scratlantis:~/ideas-ecp/TrainingCi4Ecp/build] miller86% make test
Running tests...
Test project /Users/miller86/ideas-ecp/TrainingCi4Ecp/build
Start 1: test_1
1/2 Test #1: test_1 ........................... Passed 0.00 sec
Start 2: blt_gtest_smoke
2/2 Test #2: blt_gtest_smoke .................. Passed 0.00 sec
100% tests passed, 0 tests failed out of 2
It appears to run the blg_gtest_smoke after my google test. But, isn't a passing blt_gtest_smoke a requirement in order for any of my gtest tests to run? So, am curious about a) why the order is what it is and b) is it possible to depend any gtest on a passing blt_gtest_smoke so that if that doesn't pass, it doesn't even attempt to run gtest tests?
It sounds like we have an ordering issue.
There isn't a notion of order dependence between the tests - I think we just need to add them in the right order. CTest will run all of the tests.
You could jump though (CMake) hoops to depend tests to fix the order, but really i think we should just add these smoke tests first if possible.
@white238 any ideas here?
Well, perhaps wasn't completely clear. The calc_pi I am running here is as if it was my project's test. So, it isn't running calc_pi in BLT per se. I am running my project's make test and what I see is my project's gtest getting done ahead of BLT's smoke checks.
I understand, there is no difference between running in BLT or not, I think it's all in the ordering that the tests are presented, which is how blt is included influences.
The assumption is that all unit tests are independent and the order of tests is random.
This appears to be a gtest feature (rather than a bug), see: https://stackoverflow.com/questions/12743114/how-does-google-test-make-test-sequence
EDIT: Actually, the shuffled ordering might be off by default.
@kennyweiss I think thats for test cases within a gtest-using exec, not for ctest entires?