'GaveUp' when moving from tasty-hedgehog

[vrom911]

First of all, thanks a lot for this handy library! I am using it a lot already.

However, I am facing an issue when trying to move from tasty-* family to hspec-* family. Specifically, I want to move from tasty-hedgehog to hspec-hedgehog and the process itself is straightforward. I am not changing any hedgehog specific functions or any of the Gens. But I get GaveUp in my property tests now. I manage to fix a few such tests with downgrading some ranges in generators from 0 10 to 0 5, which seems odd.

Do you know about any limitations hspec-hedgehog puts on discarding or any other limits? Would really appreciate your insights on why this regression can happen :pray:

(This is a commit with the first attempts of migration to hspec-hedgehog: https://github.com/kowainik/tomland/commit/8755e76d21ca25ddeeb2e8ddb184db8a773326b9 )

[parsonsmatt]

GaveUp occurs when the propertyDiscardLimit is hit while shrinking, as far as I can tell (see the source of hedgehog:Hedgehog.Internal.Runner). You can configure this with the maxDiscardRatio function in Test.Hspec.QuickCheck.

[vrom911]

Shrinking is not the only reason. It is mentioned in this comment on the hedgehog library issues: https://github.com/hedgehogqa/haskell-hedgehog/issues/186#issuecomment-391822861

But the most mystical part is that it was working before, and stopped working now, even though I didn't configure anything differently anywhere. The question is more about if there are some limits in hspec-hedgehog itself to make this happen?

[parsonsmatt]

The problem is that QuickCheck has a default discard ratio of 10, while hedgehog has a default discard limit of 100. I think we could make this more seamless by adding 90 to the QuickCheck configuration -> Hedgehog configuration.

[vrom911]

Thank you, this totally solves the problem.

Would you like to see the PR with the changed config where maxDiscardRatio is set to 100? Or maybe there could be a better solution for that? :thinking:

[parsonsmatt]

Well, setting it directly to 100 is bad, because people will use things like maxDiscardRatio (* 123), and having the discard ratio be 100 would be surprising. That's why I think (+ 90) is appropriate - if you expect 100 and get 10, that's bad, but if you expect 1000 and get 1090, that's not too bad. But if you expect eg < 10 and get 95, that may be bad.

So maybe what the logic should do is:

• If it is equal to 10, set to 100, and document this fact so people know to set it to like 11 if they need a smaller number of runs.
• Otherwise, leave it as-is.