plutus icon indicating copy to clipboard operation
plutus copied to clipboard

Fix and add new tests for controlling distribution of generated values

Open effectfully opened this issue 1 year ago • 0 comments

We have a number of tests showing the distribution of generated values, for example

test_listDistribution :: Int -> ([()] -> Gen [[()]]) -> Int -> TestTree
test_listDistribution numRuns split n =
    testProperty ("for a list of length " ++ show n) $
        withMaxSuccess numRuns . forAll (split $ replicate n ()) $ \aSplit ->
            label (show $ map length aSplit) True

test_multiSplitDistribution :: TestTree
test_multiSplitDistribution =
    testGroup "distribution of values generated by"
        [ testGroup "multiSplit1"
            [ test_listDistribution 10000 (coerce $ multiSplit1 @()) 1
            , test_listDistribution 10000 (coerce $ multiSplit1 @()) 2
            , test_listDistribution 10000 (coerce $ multiSplit1 @()) 3
            , test_listDistribution 10000 (coerce $ multiSplit1 @()) 4
            , test_listDistribution 10000 (coerce $ multiSplit1 @()) 5
            ]
        , testGroup "multiSplit0"
            [ test_listDistribution 1000 (multiSplit0 0.1) 1
            , test_listDistribution 1000 (multiSplit0 0.05) 2
            , test_listDistribution 1000 (multiSplit0 0.01) 3
            ]
        ]

but we don't have any tests to ensure that the distribution is sensible, you have to eyeball it.

And when we do have such tests, they usefulness is quite limited:

schnorrSecp256k1Prop :: PropertyT IO ()
schnorrSecp256k1Prop = do
  testCase <- forAllWith ppShow genSchnorrCase
  cover 5 "malformed verification key" . is (_ShouldError . _BadVerKey) $ testCase
  cover 5 "malformed signature" . is (_ShouldError . _BadSignature) $ testCase
  cover 5 "mismatch of signing key and verification key" . is (_Shouldn'tError . _WrongVerKey) $ testCase
  cover 5 "mismatch of message and signature" . is (_Shouldn'tError . _WrongSignature) $ testCase
  cover 5 "happy path" . is (_Shouldn'tError . _AllGood) $ testCase
  runTestDataWith def testCase id VerifySchnorrSecp256k1Signature

ed25519Prop :: BuiltinSemanticsVariant DefaultFun -> PropertyT IO ()
ed25519Prop semvar = do
  testCase <- forAllWith ppShow genEd25519Case
  cover 5 "malformed verification key" . is (_ShouldError . _BadVerKey) $ testCase
  cover 5 "malformed signature" . is (_ShouldError . _BadSignature) $ testCase
  cover 5 "mismatch of signing key and verification key" . is (_Shouldn'tError . _WrongVerKey) $ testCase
  cover 5 "mismatch of message and signature" . is (_Shouldn'tError . _WrongSignature) $ testCase
  cover 5 "happy path" . is (_Shouldn'tError . _AllGood) $ testCase
  runTestDataWith semvar testCase id VerifyEd25519Signature

Those cover 5 should be cover 15 or similar.

effectfully avatar Jul 25 '24 07:07 effectfully