tasty
tasty copied to clipboard
Allow test duration output to be configured
Ever since tasty-0.10
, the durations of individual tests (above the 0.01s threshold) and the overall duration of the test suite are displayed by the consoleTestReporter
ingredient. This PR adds a new option which controls which test durations are displayed:
-
--show-durations=all
displays individual and overall test durations. This is the existing behaviour and remains the default option. -
--show-durations=overall
only displays the overall test duration (in the summary at the end). -
--show-durations=tests
only displays the durations of individual tests (above the 0.01s threshold). -
--show-durations=none
displays neither.
I see two applications for wanting to control this behaviour:
- For golden tests that want to check the test output itself (e.g. when building libraries that extend
tasty
's functionality). Disabling the reporting of test durations leads to more stable output. - For teaching purposes, students often end up obsessing over test timings when they are displayed, so being able to disable them for test suites given to students may avoid some anxiety for them.
FWIW an ability to control duration output would be useful for tasty-bench
.
Please rebase onto master
and force-push to try and start CI.
@andreasabel I have rebased on master
, but you (or another maintainer) needs to approve the workflow.
I don't feel like I can make a judgement on whether this feature should be included. But I observe that no tests are included in this PR; and I think that such would be needed to accept the PR.
Sorry about the lack of tests. I had a look at the other options for the consoleTestReporter
when I implemented this new option, but couldn't find any tests for those, so there wasn't a natural place for me to add tests for this PR. The contribution guidelines for this repository also don't state that tests are required, so I thought it would be ok without them.
Let me if you have any preference for where/how they should be added and I will see what I can do when I next have some time to work on this.
Let me if you have any preference for where/how they should be added and I will see what I can do when I next have some time to work on this.
Not sure either, since I am only a steward maintainer.
There are some behavioral tests in coretests/
(shell scripts e.g.), but I don't know how they work. This is something to find out.
I notice that probably also the README needs to be updated (which is the documentation AFAICS), to include the new option.
Closing as superseded by #373. Thanks for your efforts @mbg!