ouroboros-network
ouroboros-network copied to clipboard
Unified adjustment of number of QuickCheck tests
In several places, we adapt the number of QuickCheck tests to run from the default (100), either to a higher value, e.g. for important combinatorial tests, or lower, e.g. because they otherwise take too long in CI.
The goal of this ticket is to make sure that all such occurences use tasty (via adjustOption
+ QuickCheckTests
) to do so, instead of QuickCheck's withMaxSuccess
, which is not adaptable via tasty's CLI flags.
Some context: we used to have a way to run nightly tests, via BuildKite. But then IOG removed its BuildKite infra structure, so our Test.Util.Nightly
stuff is no longer used.
-
We have a module
module Test.Util.Nightly
that defines atasty
"ingredient" for informing the individualtasty
tests whether they're being run as normal or being run at night. This is what our BuildKite script used. -
Looks like Networking has a
nightly
flag inouroboros-network-testing/ouroboros-network-testing.cabal
, so they approached that same question differently. -
I could have sworn I saw that Clarke had another approach to
Test.Util.Nightly
that I had found after the fact, but I'm unable to find it now.