🐛 Fix validation error Optional[list[...]], Union[list[...], ...], Optional[set[...]] and etc in FormData

[dotX12]

Fix issues with Optional and Union for list, set, tuple, etc... in Form:

a: Optional[list[int]] = Form(None)
b: Union[list[int], list[float]] = Form(None)

which lead to a validation error: "Input should be a valid list"

Reproduce the issue:

from typing import Optional
from fastapi import FastAPI, Form
from fastapi.testclient import TestClient

app = FastAPI()


@app.post("/form")
async def test_form(
    list_int: Optional[list[int]] = Form(None),
):
    return {"elements": list_int}


client = TestClient(app=app)


def test_send_optional_one_element():
    data = {"list_int": "100"}
    response = client.post("/form", data=data)
    assert response.json() == {"elements": [100]}

    # {'detail': [{'input': '100',
    #              'loc': ['body', 'list_int'],
    #              'msg': 'Input should be a valid list',
    #              'type': 'list_type',
    #              'url': 'https://errors.pydantic.dev/2.0.3/v/list_type'}]} != {'elements': [100]}


def test_send_optional_two_list_elements():
    data = {"list_int": ["10", "20"]}
    response = client.post("/form", data=data)
    assert response.json() == {"elements": [10, 20]}
    # {'detail': [{'input': '20',
    #              'loc': ['body', 'list_int'],
    #              'msg': 'Input should be a valid list',
    #              'type': 'list_type',
    #              'url': 'https://errors.pydantic.dev/2.0.3/v/list_type'}]} != {'elements': [10, 20]}


def test_send_optional_list_elements():
    response = client.post("/form")
    assert response.json() == {"elements": []}
    # {'elements': None} != {'elements': []}

Previously, the ModelField had a shape attribute that said what kind of field it was (list, set, tuple, etc.). So then the Optional[list[int]] shape had SHAPE_LIST

In the new version, this field (shape) no longer exists, and for the Form to parse data, FastAPI itself tries to collect values depending on the type. for list[int] iterates through the collection value=12&value=20 and does value=[12, 20]

In the new version, Optional[list[int]] does not count as a list when tested with is_sequence_field , which causes the value to be fetched as one element by : value = received_body.get(field.alias)

If we made an annotation like Optional[list[int]] then pydantic expects us to pass [1, 2, 3] or [1], but we are passing the value directly, which causes a validation error "input should be a valid list"

In simple terms, what is happening:

class Foo(BaseModel):
      a: Optional[list[int]] = Field(None)

Foo(a="1")  # raise ValidationError

And it should happen:

Foo(a=["1"])  # Ok

UPD: Initially, I decided to add field_annotation_is_optional_sequence for both V1 and V2, but after analyzing in more detail, V1 works correctly because of the shape, so field_annotation_is_optional_sequence is needed only for V2.

[dotX12]

@tiangolo, opened again, because the problem has not been fixed by anyone for a long time, please take a look :)

[dotX12]

@tiangolo Is there any information on when this will be approved and go to the master branch? I, like many other people, cannot move to >0.100 because of this problem and continue to sit at <0.99 version.

[dotX12]

@Kludex, can we engage you to solve this problem? I understand that there are a lot of problems, but the fix is important and has not moved for half a year :(

[YuriiMotov]

Just noticed that optional list parameters are not displayed correctly in Swagger. list_int: Optional[list[int]] = Form(None) is displayed as optional field without type. Even though in openapi.json its type is null or array of integer items. This problem is also relevant to Query parameters. So, I think the problem I've just mentioned should be solved separately from this PR. I'm going to create a new issue.

UPD: found similar question in discussions. It's proposed to use list_int: list[int] | SkipJsonSchema[None] = Form(None) to make Swagger understood parameter's type correctly. Ugly, but it works..

[zawsq]

Any updates to this?

As mentioned above list_int: list[int] | SkipJsonSchema[None] = Form(None) doesn't seem to work anymore, Swagger just refuse to display null value.

[dotX12]

@tiangolo, should we expect that this PR will someday end up in master? Soon it will be a year since the problem in V2 exists.