rapidcheck icon indicating copy to clipboard operation
rapidcheck copied to clipboard

Assertion is not always printed for non-deterministic tests

Open furuholm opened this issue 9 years ago • 2 comments

For tests that are not deterministic the cause of the failure will not always be printed. My guess is that this is caused by the fact that after shrinking, RapidCheck reruns the smallest failing test case and uses this for printing. If the failure is not reproduced during this run, the assertion will not be printed. This also happens when noshrink is enabled.

furuholm avatar Nov 07 '16 15:11 furuholm

Maybe this could be solved by implementing #99?

furuholm avatar Nov 10 '16 08:11 furuholm

I think you could solve this by saving the last seen actual failure result and simply use that. However, the problem is that shrinking will be slower since you will have to calculate the counterexample even if there might be an even better counterexample.

In this case, I'd lean more towards actually running the final test a bunch of times until it fails again :)

emil-e avatar Nov 10 '16 09:11 emil-e