disco
disco copied to clipboard
Redesign property generation & testing
e.g. currently desugaring happens for every single test! There should be a desugaring pass that happens before tests are run. Desugaring can also include translating special properties into universally quantified ones for randomized testing (but keeping the special names around for printing results).
The way we are returning results from testing is also silly, we should cleanly separate the results from printing messages.
This should happen as part of the Prop
extension (#217). Properties will no longer be a separate syntactic category, but just terms of type Prop
.
Well, it didn't happen (at least not completely) as part of #217 . The next step is to just look through the code and make a plan for how to clean things up.
Here is the current organization of code related to properties:
-
Disco.Property
has some utility functions for generating samples and inverting results. -
Disco.Value
has code for pretty-printing test results (prettyTestResult
, etc.). Those functions need to callDisco.Value.prettyValue
, but otherwise there doesn't seem to be any particular reason for them to be there. They should probably move to their own module which importsDisco.Value
. In fact,Disco.Property
already importsDisco.Value
. -
Disco.Interpret.CESK
definesensureProp
,testProperty
,runTest
, + a few other related utility functions. They call CESK stuff (e.g.runTest
callseval
which callsrunCESK
) and in turn CESK evaluation of e.g.holds
will calltestProperty
. -
Disco.Interactive.Commands
definesrunAllTests
, which currently returnsSem r Bool
; want to change it to return a summary of results instead of aBool
. Also hashandleTest
which callsrunTest
.- Both the above mentioned use specific numbers of examples (50 and 100) which we ought to make configurable somehow.
Edit: this has likely changed since I wrote that comment. First task would be to just get a survey of how the code is currently organized.