Error message discrepancy

[mardiros]

Initial Checks

• [X] I confirm that I'm using Pydantic V2

Description

Hi,

This is a small bug (if I can even call it a bug).

But type str and SecretStr does not return the same type on error.

"too_short" for SecretStr "string_too_short" for the native str type.

For the developper experience, I guess it is better too only have the "string_too_short" for both type.

Example Code

from pydantic import BaseModel, Field, SecretStr, ValidationError


class A(BaseModel):
    a: str = Field(min_length=2)


class B(BaseModel):
    b: SecretStr = Field(min_length=2)


def test_string_too_short():
    with pytest.raises(ValidationError) as ctxa:
        A(a="a")

    with pytest.raises(ValidationError) as ctxb:
        B(b="b")

    assert ctxa.value.errors()[0]['type'] == ctxb.value.errors()[0]['type']


>       assert ctxa.value.errors()[0]['type'] == ctxb.value.errors()[0]['type']
E       AssertionError: assert 'string_too_short' == 'too_short'
E         - too_short
E         + string_too_short

Python, Pydantic & OS Version

pydantic version: 2.6.4
        pydantic-core version: 2.16.3
          pydantic-core build: profile=release pgo=true
                 install path: /home/guillaume/.cache/pypoetry/virtualenvs/redacted-HfF0dALa-py3.11/lib/python3.11/site-packages/pydantic
               python version: 3.11.8 (main, Feb 12 2024, 14:50:05) [GCC 13.2.1 20230801]
                     platform: Linux-6.7.4-arch1-1-x86_64-with-glibc2.39
             related packages: typing_extensions-4.7.1
                       commit: unknown

[sydney-runkle]

@mardiros,

Thanks for reporting this. I agree that consistency would be good here. One issue, though, is that our version policy guarantees that error type values will stay consistent, see https://docs.pydantic.dev/latest/version-policy/#pydantic-v2. Thus, I think it probably makes sense to make this change in V3. Will tag this as such :).

[michtesar]

Hey @sydney-runkle this seems like a deeper one, for example SecretBytes also returns too_short, while bytes type return a proper message of bytes_too_short. If you want, I can take a look at it, but do you have any hint of other types might be off, so we can fix all of them at once? Also, I am having hard times to get where the validation actually happens due to a high level of abstraction, could you point me out to some example, please? Thanks!

[mishaal79]

Hello @sydney-runkle I would like to work on this issue and submit a PR.

[mishaal79]

@sydney-runkle @michtesar. I have examined the problem in the codebase and found that the discrepancy in return type is due to the use of different validators for fields that are Secrets and standard Strings. The validator used for Strings is in pydantic-core implemented in Rust which contains the specific return type of string_too_short. I'm not aware what would be a suitable fix. So any direction would be helpful.

[sydney-runkle]

@michtesar @mishaal79,

Indeed, I think reconciling the differences in terms of core schemas could be beneficial, but once again, that'll have to wait until V3 given that we promise in our versioning policy to maintain error type consistency.