qcheck
qcheck copied to clipboard
c-cube
Factor shrink loggers
This PR pulls out the shrink logging from the expect tests as it was cluttering the .expected output files needlessly and conflating
- checking basic functionality and
- highlighting the underlying shrinking algorithms (and their potential limitations).
There is now a separate directory shrink_algo_logs with a bunch of expect tests.
The idea is that these (in some cases terrifying) expect logs should gradually become nicer as each limitation is addressed 😀
One could say it turns a bunch of prior shrinking algorithm observations scattered around in comments into actual tests for everyone to see and improve on.
Highlights (many of which have been observed before):
-
char_char_never_abcdef*logs howQCheck.chardoesn't shrink (issue #166) -
fun_{first,last}_foldleftright_qcheck{,2}.expectedshows how the QCheck function shrinker's "generate-function-last-for-best-result" comes at a price of many shrink steps (room for algorithmic improvement?) and that QCheck2's function shrinker also could use a helping hand (issue #163) - the
int_*tests show QCheck2 repeatedly tests for 0 (aggressive shrinking - see below) -
list_shorter*shows how QCheck2 will try new random lists rather than cut down the first counterexample and 2 limitations of QCheck design (issue #64):- (1) it wastes many needless shrink steps to restart and try smaller sizes repeatedly
- (2) an aggressive int shrinker (trying 0 first like QCheck2) could cut down a number of steps (and restarts!)
-
string_empty*shows how QCheck doesn't shrink the string's characters (mentioned in issue #157) -
string_never_has_000*shows how QCheck2's string shrinker tries random (unrelated) strings to cut down the size, while QCheck's shrink algorithm uses a simple iterative algorithm rather than bisection or something list-shrinking-inspired (both mentioned in #157).
Finally, it
- renamed test.ml to a more descriptive name
- simplified the dune logic
- removed shrink logging from existing tests and expect-outputs
