batavia
batavia copied to clipboard
Use Hypothesis to generate example data for tests
Equivalent to pybee/voc#580.
I'm a maintainer of Hypothesis, and discussed using it to test Batavia with Russell (@freakboy3742) at the PyConAU sprints. I've got more than enough to do working on Hypothesis itself, but would be delighted to consult, mentor, teach, or assist anyone who wants to use it to test beeware things. Just @-mention me, and I'll answer!
The idea is that instead of checking predefined examples in tests/utils/sample.py:SAMPLE_DATA, you would pick the right Hypothesis strategy and get examples from that (using a test decorated with @given, so examples are reproducible and minimize correctly).
As a quick-and-dirty demo, we could also just (temporarily) replace SAMPLE_DATA with a dataset drawn from Hypothesis:
from hypothesis.strategies import from_type
SAMPLE_DATA = ... # current definition
generated = {
k: [from_type(eval(k)).example() for _ in range(100)
if isinstance(eval(k), type)]
for k in SAMPLE_DATA
}
SAMPLE_DATA = {
k: sorted(set(repr(x) for x in v), key=lambda x: len(x), x)
for k, v in generated.items()
}
This wouldn't show minimal examples, but it would probably turn up a bunch of unicode issues and doesn't require any changes to existing tests.