pydantic
Hypothesis plugin does not work for ConstrainedStr
@Zac-HD's Hypothesis plugin https://github.com/samuelcolvin/pydantic/pull/2097 works great for some constrained type (e.g. ConstrainedInt), but not for others (e.g. ConstrainedStr):
from pydantic import ConstrainedInt, ConstrainedStr
import hypothesis.strategies as st
class MyInt(ConstrainedInt):
gt = 10
lt = 15
class MyStr(ConstrainedStr):
regex = "[A-Z][0-9]{10}"
st.from_type(MyInt).example() # As expected, values are between `gt` and `lt`
st.from_type(MyStr).example() # Returns empty str :-(
Thanks for the awesome work by the way @Zac-HD!
Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":
pydantic version: 1.8.1
pydantic compiled: True
install path: /Users/.../lib/python3.8/site-packages/pydantic
python version: 3.8.8 (default, Feb 24 2021, 13:46:16) [Clang 10.0.0 ]
platform: macOS-10.16-x86_64-i386-64bit
optional deps. installed: ['dotenv', 'email-validator', 'typing-extensions']
Ah, right - that's because the registration hook (_registered()) for strings is in the constr() helper function, whereas for ConstrainedInt I was able to attach it to the ConstrainedNumberMeta metaclass.
So MyStr = constr(regex="[A-Z][0-9]{10}") is your workaround.
If explicit inheritance from ConstrainedStr et al is supported as part of the public API, I think the best option is to add a new metaclass for all constrained types which ConstrainedNumberMeta can then inherit from.
I believe subclassing Constrained* is preferable over using con* because of https://github.com/samuelcolvin/pydantic/issues/975: mypy will complain that T = con*(...) is not a valid type.
I tried using constr instead of ConstrainedStr and Hypothesis can indeed generate valid examples of it, but not when it's used inside of a BaseModel:
from pydantic import BaseModel, constr
import hypothesis.strategies as st
MyStr = constr(regex="[A-Z][0-9]{10}")
class MyModel(BaseModel):
foo: MyStr
st.from_type(MyStr).example() # Generates valid strings
st.from_type(MyModel).example() # Fails with a ValidationError :-(
It also appears that other constrained types such as ConstrainedList and ConstrainedSet don't work yet either, probably for the same reasons:
from pydantic import ConstrainedList, ConstrainedSet
import hypothesis.strategies as st
class NonEmptyList(ConstrainedList):
__args__ = [int]
item_type = int
min_items = 1
class NonEmptySet(ConstrainedSet):
__args__ = [int]
item_type = int
min_items = 1
st.from_type(NonEmptyList).example() # Returns [] :-(
st.from_type(NonEmptySet).example() # Returns NonEmptySet() :-(
I believe subclassing
Constrained*is preferable over usingcon*because of #975: mypy will complain thatT = con*(...)is not a valid type.
I don't have any objection to this, but so far as I know all the docs show the latter T = con*(...) style and it's not clear that explicit inheritance is meant to be supported. Either way, expanding the docs to say would be good.
I tried using
constrinstead ofConstrainedStrand Hypothesis can indeed generate valid examples of it, but not when it's used inside of aBaseModel:
This is bizzare, and apparently also under-tested 😥
Fixing this would have to start with some detailed investigation that I don't really have time for this week...
It also appears that other constrained types such as
ConstrainedListandConstrainedSetdon't work yet either, probably for the same reasons
Different reasons, actually - a particular subtype of e.g. ConstrainedList is a parametrized generic type, and Hypothesis' type-to-strategy resolution machinery only deals in non-parametrized generics (by registering a function which takes a parametrized type object and returns a strategy).
Which would be fine, except that per https://github.com/samuelcolvin/pydantic/pull/2097#issuecomment-753603121 we're waiting on https://github.com/cython/cython/issues/3537!
At present, you just have to provide (or register) a strategy for each model that uses a constrained list or set field. Maybe the docs should explicitly list what the plugin doesn't handle, too?
Also bumped into this. As already mentioned, the problem is that mypy doesn't approve of = constr() (dynamic type assignment). So you need to sub-class types. Which then doesn't work with hypothesis, currently.
This should be mentioned in the docs, at least.
I was able to workaround this by manually calling pydantic.types._registered() for my sub-types, like:
class Str1(StrictStr):
min_length = 1
pydantic.types._registered(Str1)
Alternative would be to register them via a decorator of a metaclass, but nah, this is the simplest solution to begin with.
I'm seeing the same issue with conlist. Hypothesis is unaware of the min/max length.
A year later, can we have an update on this ? Is this going to be solved ? Or should we go with the workaround ? I think it's a critical feature to be developed
PS : What @tuukkamonsten is proposing works partially. It takes into account the min_lenght constraint for example. However it still does not work for regex constraints
Still blocked on cython/cython#3537
Looks like that is fixed now, or?
Unfortunately while the fix has been merged, it still hasn't been released yet and therefore doesn't help us. This is definitely one of the design considerations for Pydantic v2, but I think probably can't be fixed before then 😥
We’re no longer actively developing Pydantic V1 (although it will continue to receive security fixes for the next year or so), so we’re closing issues solely related to V1. If you really think this shouldn’t be closed, please comment or create a new issue 🚀.
