conformance icon indicating copy to clipboard operation
conformance copied to clipboard

enhancement: ability to trace known failing or even passing test cases

Open jhump opened this issue 1 year ago • 1 comments

Currently, the --trace flag enables traces for unexpected failing test cases. But it can be useful to see traces for known-failing test cases. And it can even be useful sometimes to see the traces for passing test cases -- like to compare two different implementations and see what a "good" trace looks like vs. a "bad" trace for a test case failure.

An easy way to add this into the current command-line interface could be with two new flags:

  1. --trace-known-failing: causes traces for known-failing and known-flaky test cases to be shown
  2. --force-trace <pattern>: causes traces to be shown for test cases that match the pattern, regardless of whether they pass or fail.

jhump avatar Jun 14 '24 20:06 jhump

I would find traces for passing tests very useful. IMO it's sufficient to enable tracing for all outcomes for all tests with --trace --vv.

timostamm avatar Sep 11 '24 11:09 timostamm