quickcheck-state-machine
quickcheck-state-machine copied to clipboard
Add pretty print for CounterExample and improve test output
tests print things like
AnnotateC "Read" (PredicateC (1 :/= 2))
PostconditionFailed "AnnotateC \"Read\" (PredicateC (6 :/= 5))" /= Ok
which may not be very helpful, or just hard to understand for the user
Also drawing for CrashAndLogic is broken for some reason.
Yes, this can be improved for sure!
I once had the idea that a nice output could be obtained by combining the initial Logic formula and the Counterexample that the formula evaluated to, e.g. something like:
prettyPrintCounterexample :: Logic -> Counterexample -> String
prettyPrintCounterexample l@(p :&& q) (Fst ce) = concat
[ "When checking that: "
, show l
, " is true, "
, "the first component is false, because: "
, prettyPrintCounterexample p ce
]
eval :: Logic -> Either String ()
eval l = case logic l of
VTrue -> Right ()
VFalse ce -> Left (prettyPrintCounterexample l ce)
I think the tricky part is how to not be too verbose and yet precise. Perhaps underlining the component of the formula that was false? Perhaps one should check how deep the formula is and only show parts of the tree? What do other tools do?