kaocha
kaocha copied to clipboard
Interpret the "0 test" situation as a failure
Currently tests pass and no fails are thrown when there are 0 tests.
0 tests, 0 assertions, 0 failures.
Not sure if this kind of kaocha configuration is possible in the tests.edn
, but it would be useful to have the kaocha fail when 0 tests are detected. This would help indicate that configurations are incorrect, similar to how an error is thrown, if a deftest doesn't contain any assertions. This will ensure that any mis-configurations can be brought to attention
Recommendation on Slack, by @lread :
@onetom, I would prefer a non zero exit in this scenario as well. But... it is also pretty easy to write a kaocha hook to achieve this.
Thanks for suggesting this improvement! I think this makes a lot of sense. Maybe not as default behavior, but as an option you can enable.
This could be a good candidate for https://github.com/lambdaisland/kaocha/issues/183 , we can already introduce a flag to opt-in, then change the default for #kaocha/v2
Along similar lines, the No tests found
situation might also be considered a failure.
https://github.com/lambdaisland/kaocha/blob/3d834940d420d6490a2290f809526fff07ef7775/src/kaocha/api.clj#L104-L106
That might also make internal bugs like https://github.com/lambdaisland/kaocha/issues/208 more apparent.
It seems that it can be related the PR https://github.com/lambdaisland/kaocha/pull/404