fast-check icon indicating copy to clipboard operation
fast-check copied to clipboard

Print `verbose: 2` execution summary during the run

Open gruhn opened this issue 7 months ago • 1 comments

🚀 Feature Request

How much trouble would it be to print the verbose: 2 execution summary during the test run, instead of printing it in batch at the end? Using the example from the documentation:

Execution summary:
× [[2147483632,-1868311174,2,-14,-1309756521,948824737,2147483631,1352827217]]
. √ [[]]
. × [[-1309756521,948824737,2147483631,1352827217]]
. . √ [[2147483631,1352827217]]
. . × [[948824737,2147483631,1352827217]]
. . . √ [[2147483631,1352827217]]
. . . √ [[0,2147483631,1352827217]]
. . . × [[474412369,2147483631,1352827217]]
. . . . √ [[2147483631,1352827217]]
. . . . × [[237206185,2147483631,1352827217]]
. . . . . √ [[2147483631,1352827217]]
. . . . . √ [[118603093,2147483631,1352827217]]
. . . . . × [[177904639,2147483631,1352827217]]
...
. . . . . ... . . × [[2,1000000000]]
. . . . . ... . . . √ [[1000000000]]
. . . . . ... . . . √ [[1,1000000000]]
. . . . . ... . . . √ [[2]]
. . . . . ... . . . √ [[2,999999999]]

Presumably, each of these lines can (in principle) be printed as soon as the corresponding test has failed/succeeded. The level of indentation is also known, right? I know there is some effort going on with workers to run tests in parallel? Would that be a problem here?

Motivation

This is just a minor improvement from my view. And I guess for fast running tests, this would make no difference. I have some fairly long running (model-based) tests though. There it would be nice to have more lively feedback. In particular during shrinking. Sometimes shrinking takes many iterations. I noticed that team members, who are not so familiar with the tests, get the impression that the tests are "stuck". Recently, I also gave a presentation on our testing setup with a live demo. I started the tests, then bridged 2 minutes with other talking points, and finally explained the idea of shrinking, once the execution summary was available. No problem but it also would have been slightly nicer to explain the shrinking process as it was happening.

gruhn avatar Jun 28 '24 22:06 gruhn