qcheck icon indicating copy to clipboard operation
qcheck copied to clipboard

Deriver: derive shrinkers

Open vch9 opened this issue 4 years ago • 5 comments

We could derive shrinker from the type declaration. However, shouldn't we admit that people wanting shrinkers should migrate to QCheck2 instead? ping @jmid as you mentioned this

vch9 avatar Oct 11 '21 13:10 vch9

I think it's still not clear that QCheck2 will always be the better choice. A shrinker generator would be useful. However it's not urgent, I think it's reasonable to focus on QCheck2 generators instead and do a release.

c-cube avatar Oct 11 '21 16:10 c-cube

I agree with @c-cube. The QCheck2 shrinker code is still relatively immature.

Also: as I tried to illustrate in the shrink logs #172 QCheck2 sometimes has shrinker issues due to missing a splittable RNG + sub-optimal shrinker algorithms. QCheck's shrinkers are not optimal either (description in #177) - but with a couple of improvements to the list and string shrinkers (lying on my HD in a ~1 month old git stash... 😅) all the benchmarks run reasonably fast. I hope to push these at some point - and that the algorithms can then be mirrored in QCheck2...

As to a splittable RNG I've suggested we go with a "poor man's" implementation first. I noticed we already use such an approach in the runner to check a list of tests independently of each other: https://github.com/c-cube/qcheck/blob/115de55147beaf23ca5761f37bacd23716a28f2c/src/runner/QCheck_base_runner.ml#L389

jmid avatar Oct 11 '21 17:10 jmid

If we plan on adding the shrinker deriver in the ppx, we should decide before any release the plugins names: #190.

vch9 avatar Oct 12 '21 07:10 vch9

@jmid Have you tried either of pringo or splittable_random as a splittable RNG library?

tmcgilchrist avatar Jan 19 '22 07:01 tmcgilchrist

@jmid Have you tried either of pringo or splittable_random as a splittable RNG library?

I think I tried them briefly at some point. My interest was in how splitting enabled more predictable/better shrinking in the rose-tree setup by decoupling random choices (and thus shrink trees) of sub-parts. (rereading that I'm not sure anyone understands 😅)

There's big news in the pipeline for 5.00 though: https://github.com/ocaml/ocaml/pull/10742 So

  • we should start preparing for a QCheck setup that can work on both pre-5.00 and 5.00 with split
  • the splitting per domain enables a "parallel QCheck mode" where several cores run in parallel 😀

jmid avatar Jan 19 '22 08:01 jmid