snitch

snitch copied to clipboard

Currently, if a test calls std::terminate(), the whole test application stops. We should be able to register a handler for the abort signal (doctest does this), report the failure, and continue with the next test.

We need to be able to figure out who calls std::terminate(); if the test framework itself calls it (because of out-of-bounds access, or reaching a hard-coded limit), we should propagate the error and stop. Ideally we would implement this so that we can test for our own termination; perhaps the test runner function could register the abort handler just before starting the test?

At any rate, this would allow us to support REQUIRE_* and FAIL when exceptions are not used, if we make them use std::terminate() instead of return.

Apparently, "catch and continue" is impossible within the same process. We would need to run each test into a sub-process, which is what Google Test does to support "death tests". At the time of writing this, neither Catch2 nor doctest support this, while Boost UT has an implementation for Unix only (no Windows).

https://github.com/cschreib/snatch/pull/38 will nonetheless switch REQUIRE() to call std::terminate() when exceptions are disabled, because return is not reliable enough (when called in functions and not in test cases). Adding support for "catch and continue", or improving reporting on caught signals, will be postponed to a future version (but possibly may never happen...)