Can we un-obsolete this? I have use cases.

[kindaro]

I am not ready to agree that this library is useless and falsify or other randomized checking library should be used instead. I have two use cases unfit for randomized checking. In these use cases, the issue with randomized checking is not that counterexamples are too big. Rather, my use cases require two qualities only SmallCheck and no randomized checking library offers: totality on small examples and immediate access to series.

mathematical properties

This example illustrates the benefit of totality on small examples.

Say you are defining a new group or ring. You surely want that all properties are checked with the additive and multiplicative identity element. But randomized checking does not guarantee this. I have observed this in practice, where my supposed group had a faulty identity — in some runs, the fault will be detected, in other runs, it will be missed. Changing the size and the number of examples checked does not help — the additional examples will only get bigger and bigger, and the identity element is a small one. With SmallCheck, this is not an issue — all the important small elements are always among the first to be generated.

unambiguous parsing

This example illustrates the benefit of immediate access to series.

Say you are defining a language and you wish to have an unambiguous grammar. One approach is to derive a parser and a generator from your grammar, and then see that anything generated is parsed unambiguously. For example, in Haskell, Earley is a library that can automatically derive both a parser and a generator for a given grammar. The function that generates all strings in the language of length up to some number is at once a SmallCheck series, while defining a random generator of strings of a given language may be a non-trivial task of its own, either because of the danger of random combinatorial explosions or because, as in the case of Earley, the library used to define grammars does not provide easy access to the grammar itself, but only offers ready-made parsers and generators.

[Bodigrim]

I'm well aware of a usual talk in favor of smallcheck, but over the years I've seen very few specific examples where it is superior to QuickCheck / hedgehog / falsify. If you are in 1% of clients whose use case is better served by smallcheck, good for you, but I'm not convinced it holds for the majority.

Dunno, over the years I tested a lot of algebraic laws for groups, rings and such and I do not recall any issues with QuickCheck generators. It would be interesting to look at your example.

As for series, the usual problem is that their length grows exponentially. You'll be out of memory well before generating any non-trivial input string. Again, I'd like to see a practical test case, where QuickCheck fails to deliver.

FWIW I think Earley can be adapted to generate random strings by choosing a random element of next in GenerationEnv.

[kindaro]

Haskell is used by 1.6% of professional software developers.

Here is a practically occurring problem which QuickCheck detects only about 19 times out of 20. SmallCheck always detects it.

main = do
	let n = minBound :: Int
	quickCheck \ x -> x /= 0 Quick.==> n `div` x * x + n `mod` x == n
	smallCheck 10 \ x -> x /= 0 Small.==> n `div` x * x + n `mod` x == n

Here is a definition of a SmallCheck series from an Earley grammar:

series_of_grammar ∷ (∀ r. Grammar r (Prod r x Char outputs)) → String → Series m (outputs, String)
series_of_grammar grammar tokens = SmallCheck.generate \depth → Earley.upTo depth (Earley.generator grammar tokens)

At present, QuickCheck does not offer anything like this.

[Bodigrim]

Here is a practically occurring problem which QuickCheck detects only about 19 times out of 20.

I think 95% is good enough, if it eliminates a need in another testing framework.

At present, QuickCheck does not offer anything like this.

Trivially a QuickCheck generator can just pick up a random element of Earley.upTo depth. A better implementation would involve trimming GenerationEnv at each step to keep only one (arbitrary) continuation.

---

I'm not sure what's exactly the ask here. Is there anything preventing you from using smallcheck at the moment?

[kindaro]

I am not talking to you in the spirit of a client requesting to be served. I am talking to you in the spirit of altruistic collaboration. The way you, as a maintainer, presented SmallCheck in the first page of documentation does not do justice to its value as a deterministic property checker. The world will become a tiny little bit better if you rephrase the documentation.

[kindaro]

That is not to say that you should not mention randomized property checkers. It is always good to link to alternatives and suggest best practices. All I am saying is that SmallCheck has the distinct advantages of a deterministic property checker, and the current documentation does not reflect that.

[Bodigrim]

The way you, as a maintainer, presented SmallCheck in the first page of documentation does not do justice to its value as a deterministic property checker.

I guess we just have to agree to disagree. As a sole active maintainer, I believe that the description is justifiable. Sorry, I don't think that discussing this disagreement further will be productive.

That is not to say that you should not mention randomized property checkers. It is always good to link to alternatives and suggest best practices. All I am saying is that SmallCheck has the distinct advantages of a deterministic property checker, and the current documentation does not reflect that.

I've added a link to the comparison to QuickCheck. I don't mind linking other good quality materials if there are any.

[kindaro]

What evidence would have persuaded you?