kaocha icon indicating copy to clipboard operation
kaocha copied to clipboard

Allow configuring skip-meta / focus-meta at the top level

Open plexus opened this issue 6 years ago • 2 comments

Currently the skip/focus configuration is part of the test suite. I think at least for the -meta versions we should also allow configuring them at the top level configuration.

This already kind of works, you can use kaocha.filter/focus-meta or kaocha.filter/skip-meta, but it's undocumented, and the shorthand versions only work in the test suite (:focus-meta and :skip-meta).

Also currently if either is specified on the command line then it will overrule what's configured, this is fine for --focus-meta I think, but not for --skip-meta, which should be additive.

plexus avatar Jan 23 '19 08:01 plexus

What's the use case of :focus-meta on the top level? That would essentially mean you'd have to essentially opt-in to a test being run by default, right?

I suppose you could have a setup where only fast/critical tests get the :run-always meta and the top-level :focus-meta contains :run-always. The entire test suite is run less often, by the separate :run-all and :ci profiles, which don't have :focus-meta.

It would be great if whoever takes this issue adds documentation explaining not only how these options work but also their use cases and maybe even instructions on how to set up something like what I just outlined. But if they're not able to, we could do it as a separate PR.

alysbrooks avatar Dec 09 '20 23:12 alysbrooks

With --focus-meta / :focus-meta, when no tests are found it will run all tests. This is somewhat deliberate, this allows you to configure :focus-meta [:focus], and when you want to just work on a single test or a few related tests you mark those with ^:focus as long as you are working on them. Especially nice with --watch I think, since you can opt in or out of running certain tests without having to restart kaocha.

We do warn when that happens, since it might also mean you just have a typo or something

WARNING: No tests found with metadata key :foobar. Ignoring --focus-meta :foobar.

Probably not a big deal, but people who use this feature a lot might want to disable this warning. I think eventually we might want to revisit our warnings and make most of them opt-out-able via tests.edn.

The other reason this could/should work is that Kaocha is also an API for use within other tooling, so your config may be programmatically generated. Although as mentioned that is not a problem here since we do allow configuring it at the top level, but the #kaocha/v1 reader macro does not normalize the keyword.

So concretely in terms of implementation I think this is mainly something to be fixed in kaocha.config/normalize (line 58 and following). Get :focus-meta / :skip-meta, when present assoc them back on as :kaocha.filter/focus-meta / :kaocha.filter/skip-meta, and then dissoc the shorthand form.

plexus avatar Dec 10 '20 15:12 plexus