penumbra icon indicating copy to clipboard operation
penumbra copied to clipboard

Use higher property testing iteration counts in CI/CD

Open cronokirby opened this issue 1 year ago • 0 comments

Is your feature request related to a problem? Please describe. Currently, for faster development iteration, we use a limited amount of iterations for our property tests. This makes tests run faster, but also means that we might be missing out on finding bugs. This constraint doesn't exists as much in CI/CD, where we can afford to have a longer running test suite.

Describe the solution you'd like We could configure the way we run our test suite in CI to allow for more iterations in our property tests, thus catching more bugs.

Now, one issue is that this might cause PRs to take longer to merge, so we might want to consider not always running this more intensive test-suite?

We should also have a neatly packaged way of managing stuff like this, like some kind of configuration flag that gets passed around: not sure what the best solution here is.

cronokirby avatar Jul 26 '23 19:07 cronokirby