weaver-test icon indicating copy to clipboard operation
weaver-test copied to clipboard

Contrib module

Open keynmol opened this issue 4 years ago • 1 comments

Weaver's commitment to talking in terms of CE's primitives is a very powerful idea, meaning that a lot of interesting things can be implemented without ever hooking into the framework's internals/changing its behaviour.

This issue is in part inspired by the following discussion: https://github.com/disneystreaming/weaver-test/discussions/374, but also by my long-standing desire to write a module for speculative testing in weaver.

So my proposal is having a contrib module, which can be seen as a collection of various ideas, strategies for testing, etc.

Constraints:

  • As much as possible, works across all of the supported effect types. Will mean artifacts like weaver-contrib-cats, weaver-contrib-monix, etc.
  • CE2 and CE3 cross-publishing, same concept as weaver (CE3 is 1 minor above, CE2 is in a terminal version line)
  • No dependencies other than relevant weaver version.
    • This might seem strange for certain things, like not using cats-retry, but we're already tied to release cycle/hygiene of effect monad maintainers, and I'd rather we don't increase the amount of dependencies
    • It will also allow us to implement more specific/obscure features (see discussion above) because it will have lower cost this way than if it had additional dependencies on top of weaver

So far my ideas are:

  1. Multiplexing:
    • Run N instances of same test, and
      • guarnatee all succeeded
      • guarantee at least 1 succeeded
      • report the first one that finished, no matter how
    • Run test until it succeeds, sequentially (with max N tests spawned)
  2. Support for "cancel all tests if one has failed" - which I didn't think of before, but now I like the idea more and more

keynmol avatar Sep 02 '21 21:09 keynmol

I agree. I think we can try to make these modules generic respectively to effect types, before defaulting to effect-specific versions...The scalacheck integration gives a good example of how to do that.

Baccata avatar Sep 02 '21 21:09 Baccata