Catch2 icon indicating copy to clipboard operation
Catch2 copied to clipboard

marking tests as xfail

Open LecrisUT opened this issue 1 year ago • 0 comments

In pytest, xfail(strict=false) (default xfail) the tests are run, but the test_suite does not fail if the individual test fails. This is different from skipping tests or negating them in that in most cases it should pass, but due to architecture, floating point errors or other kind of noises, the tests can fail.

And related to that, having a parametrized/generator tests that we expect at least one test to pass, e.g. in the above situation we can add some random noise so that the tests eventually pass.

Use-case: Tests that check if an error is thrown in an unstable regime.

LecrisUT avatar Jun 15 '23 05:06 LecrisUT