conformance
conformance copied to clipboard
enhancement: ability to trace known failing or even passing test cases
Currently, the --trace flag enables traces for unexpected failing test cases. But it can be useful to see traces for known-failing test cases. And it can even be useful sometimes to see the traces for passing test cases -- like to compare two different implementations and see what a "good" trace looks like vs. a "bad" trace for a test case failure.
An easy way to add this into the current command-line interface could be with two new flags:
-
--trace-known-failing: causes traces for known-failing and known-flaky test cases to be shown -
--force-trace <pattern>: causes traces to be shown for test cases that match the pattern, regardless of whether they pass or fail.
I would find traces for passing tests very useful. IMO it's sufficient to enable tracing for all outcomes for all tests with --trace --vv.