WIP: Refactor configuration loading to reduce duplicated code

[alysbrooks]

There's duplicated config logic across kaocha.runner and kaocha.watch.

In addition to simplifying maintenance, this change would:

• Avoid future issues where configuration loading diverges in kaocha.runner and kaocha.watch, like #310.
• Provide an easy way to work with configuration at the REPL.
• Allow watch to display errors when an invalid configuration is reloaded.

Relatedly, we may be able to address this TODO:

   ;; TODO: we're calling the config hook here in multiple places, and it's also
  ;; being called in `kaocha.api`. Given that we already need the fully expanded
  ;; config here (since now we're potentially adding suites in the config hook),
  ;; we should call it once at the top here, and pass the processed config into
  ;; kaocha.api. Punting on that because it requires a coordinated update in
  ;; kaocha.repl and kaocha-boot.

I suspect when kaocha.watch was first created it was easy enough to keep these two places in sync and most of the complexity has been added since.

[codecov[bot]]

Codecov Report

Base: 75.23% // Head: 75.47% // Increases project coverage by +0.23% :tada:

Coverage data is based on head (2747f2f) compared to base (c530053). Patch has no changes to coverable lines.

:exclamation: Current head 2747f2f differs from pull request most recent head ea9298d. Consider uploading reports for the commit ea9298d to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #315      +/-   ##
==========================================
+ Coverage   75.23%   75.47%   +0.23%     
==========================================
  Files          51       51              
  Lines        2742     2732      -10     
  Branches      259      255       -4     
==========================================
- Hits         2063     2062       -1     
+ Misses        517      510       -7     
+ Partials      162      160       -2     

Flag	Coverage Δ
integration	57.04% <0.00%> (+0.04%)	:arrow_up:
unit	69.50% <0.00%> (+0.18%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
src/kaocha/type/ns.clj	95.74% <0.00%> (-2.13%)	:arrow_down:
src/kaocha/type/var.clj	83.33% <0.00%> (-1.29%)	:arrow_down:
src/kaocha/plugin/gc_profiling.clj	37.17% <0.00%> (-0.49%)	:arrow_down:
src/kaocha/plugin/notifier.clj	82.82% <0.00%> (ø)
src/kaocha/plugin/capture_output.cljc
src/kaocha/plugin/capture_output.clj	100.00% <0.00%> (ø)
src/kaocha/watch.clj	78.30% <0.00%> (+4.44%)	:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

[alysbrooks]

We can probably handle #319 in this same PR.

[plexus]

I'd be careful here of making things too inflexible. E.g. the warning about not having a "tests.edn" makes sense when calling kaocha.runner, but does it make sense when using kaocha.repl? Do we want to print it every time when calling watch?

Having a certain amount of repetition between these different call sites serves a purpose, it means each can specialize and evolve in the ways that make most sense there, without impacting the others. So instead of striving for a single function that does everything, I would prefer we strive for having clear and composable building blocks. We currently have a fairly good separation of policy vs mechanisms. Config provides mechanism, runner/repl/watch/exec implement the policy. The main way to improve that is to make the mechanism code such that the policy code becomes clear, explicit, and precise. It should show at a high level exactly the steps its taking. We don't want to sweep that process under the rug by simply moving it to config.

[alysbrooks]

That makes a lot of sense.

Summarizing the different "policies" for configuration taken by the different entry points:

Entrypoint\Feature	Validates?	Accepts Profiles?	Merges commandline options?	Takes extra opts?	Reloads with Plugin CLI?
regular CLI (kaocha.runner/-main*)	Yes	Yes	Yes	No	Yes
dep.tools -X (kaocha.runner/exec-fn)	Added (4b16668bb48247035dccdbeedd7d4a0cbfd7fd06) ~~[0]~~	No? [1]	Yes (as extra opts)	Yes (from CLI)	Yes
REPL (kaocha.repl/config)	Yes	Added	No	Yes	No [2]
Watch mode (kaocha.watch/reload-config)	Added	Yes	Yes	No	No (already done by runner)

Added means it was added in this PR.

[0], [1], and [2] mark inconsistencies. ~~[0] seems like it should be fixed (and I may do that next)~~ [0] has been fixed, but I'm less sure about [1] and [2].

Whether to normalize regular configuration coming from files is a policy controlled by the #kaocha and #kaocha/v1 tags.

[alysbrooks]

Made those changes. I think we're converging on the right solution, but Joshua said he'd have other suggestions. Hopefully, that'll be the last round of changes. :)

[plexus]

Seems like this is a big improvement, let's get this shipped. For the remaining inconsistencies let's open a new issue for discussion.

@alysbrooks it seems you have a conflict, can you rebase?

[plexus]

Released in v1.76.1230

[lambdaisland/kaocha "1.76.1230"]                 ;; deps.edn
{lambdaisland/kaocha {:mvn/version "1.76.1230"}}  ;; project.clj