Subclass of generic model validate failed

[st1020]

Initial Checks

• [X] I confirm that I'm using Pydantic V2

Description

Just like the sample code below, I thought all validation should pass, but it fails.

Is this by design or a bug?

Example Code

from typing import Any, Generic, TypeVar

from pydantic import BaseModel, TypeAdapter, ValidationError

T = TypeVar("T")


class A(BaseModel, Generic[T]):
    pass


class B(A[Any]):
    pass


class C(A[int]):
    pass


class D(A[bool]):
    pass


# validate success
TypeAdapter(A[Any]).validate_python(B())
TypeAdapter(A[int]).validate_python(C())

# validate failed
try:
    TypeAdapter(A[Any]).validate_python(C())
except ValidationError as e:
    print(e)

# validate failed
try:
    TypeAdapter(A[int]).validate_python(D())
except ValidationError as e:
    print(e)

# validate failed
try:
    TypeAdapter(C).validate_python(D())
except ValidationError as e:
    print(e)

Python, Pydantic & OS Version

pydantic version: 2.4.2
        pydantic-core version: 2.10.1
          pydantic-core build: profile=release pgo=false
                 install path: /usr/local/lib/python3.11/site-packages/pydantic
               python version: 3.11.6 (main, Oct  3 2023, 02:51:45) [Clang 14.0.3 (clang-1403.0.22.14.1)]
                     platform: macOS-13.5.2-x86_64-i386-64bit
             related packages: mypy-1.5.1 typing_extensions-4.8.0

[sydney-runkle]

Hi @st1020,

Thanks for reporting this. Our generic models docs issue a warning about isinstance checks with parametrized generics. The docs state:

Based on the nature of your example, I think that's the main issue that you're running into here. Let me know if you have any other questions!

[st1020]

Yes, I noticed the warning, but I had thought there would be some different behavior using TypeAdapter.

Is there a way to get the unsubscripted version and type arguments of a generic model, like typing.get_origin() and typing.get_args() do?

Thanks!

[dmontagu]

@st1020 we have pydantic._internal._generics.get_args and pydantic._internal._generics.get_origin.

Obviously with the _internal you can see they are private (and subject to change), but I highly doubt this will change significantly before v3. I am hoping that as part of a future version of pydantic we can make pydantic generics behave much more like standard library generics so that the typing.get_args and typing.get_origin functions will "just work", but there's still some work we'd need to do to make that work.

(Open to a PR if anyone can figure out how to make that work without breaking changes; it would require making parametrized submodels be subclasses of typing._GenericAlias, I have tried some things in the past but not been able to resolve all the challenges.)

[dmontagu]

The specific validation errors you are getting a consequence of the fact that parametrizing a pydantic model creates an actual subclass, so these are correct. More generally, there is no way to check "subtype" relationships using instance checks for covariant models with standard library generics, so I wouldn't expect to find a good way to do your:

TypeAdapter(C).validate_python(D())

check at runtime (short of implementing different logic than subclass checking for the validation).

I have strongly considered reworking pydantic's subclass validation stuff to allow differences in the generic parameters when validating, and if they aren't exactly the same, just revalidate as long as the generic origin is the same. And I'm not saying that we can't/shouldn't do that eventually, but this leads to so many edge cases to consider that it's hard to feel confident that that would be safe in practice in all scenarios.

In general, open to improvements here. I'll also note that the use of a mode='before' @model_validator may help you to manually resolve some of these issues, depending on what you want to do. (E.g., revalidate the dumped model dict if the generic origin is the same, even if the parameters are not.)

[vkcku]

I have tried some things in the past but not been able to resolve all the challenges.

@dmontagu could you expand on this a bit? I faced the issue of get_args not working as expected (#7837) and wanted to see if this could be fixed in pydantic itself instead of having to maintain a separate pydantic_get_args.

[st1020]

@dmontagu Thank you for your detailed answer!