prefect icon indicating copy to clipboard operation
prefect copied to clipboard

TypeError raised during handling of Validation Error

Open ndamclean opened this issue 7 months ago • 1 comments

First check

  • [X] I added a descriptive title to this issue.
  • [X] I used the GitHub search to find a similar issue and didn't find it.
  • [X] I searched the Prefect documentation for this issue.
  • [X] I checked that this issue is related to Prefect and not one of its dependencies.

Bug summary

Prefect seems to have a bug in it's validation exception handling code where an error is raised during validation exception handling if the parameter being validated is a list of Pydantic models.

This confuses the error logs and makes it harder to see the actual validation error.

Reproduction

"""
Call this flow with the following parameters in order to reproduce the issue:
{
  "params": [{"name": "rodrigo"}, {}]
}
"""
from prefect import flow

from pydantic import BaseModel


class HelloParams(BaseModel):
    name: str


@flow(log_prints=True)
def hello(params: list[HelloParams]):
    for param_set in params:
        print(f"hello {param_set.name}!")


if __name__ == "__main__":
    hello.serve()

Error

Traceback (most recent call last):
  File "/home/nmclean/.cache/pypoetry/virtualenvs/global-calculator-UYA1GDje-py3.11/lib/python3.11/site-packages/prefect/flows.py", line 536, in validate_parameters
    model = validated_fn.init_model_instance(*args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nmclean/.cache/pypoetry/virtualenvs/global-calculator-UYA1GDje-py3.11/lib/python3.11/site-packages/pydantic/v1/decorator.py", line 130, in init_model_instance
    return self.model(**values)
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/nmclean/.cache/pypoetry/virtualenvs/global-calculator-UYA1GDje-py3.11/lib/python3.11/site-packages/pydantic/main.py", line 171, in __init__
    self.__pydantic_validator__.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for Hello
params.1.name
  Field required [type=missing, input_value={}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.6/v/missing

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/nmclean/.cache/pypoetry/virtualenvs/global-calculator-UYA1GDje-py3.11/lib/python3.11/site-packages/prefect/engine.py", line 454, in retrieve_flow_then_begin_flow_run
    parameters = flow.validate_parameters(flow_run.parameters)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nmclean/.cache/pypoetry/virtualenvs/global-calculator-UYA1GDje-py3.11/lib/python3.11/site-packages/prefect/flows.py", line 544, in validate_parameters
    raise ParameterTypeError.from_validation_error(exc) from None
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nmclean/.cache/pypoetry/virtualenvs/global-calculator-UYA1GDje-py3.11/lib/python3.11/site-packages/prefect/exceptions.py", line 181, in from_validation_error
    bad_params = [f'{".".join(err["loc"])}: {err["msg"]}' for err in exc.errors()]
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/nmclean/.cache/pypoetry/virtualenvs/global-calculator-UYA1GDje-py3.11/lib/python3.11/site-packages/prefect/exceptions.py", line 181, in <listcomp>
    bad_params = [f'{".".join(err["loc"])}: {err["msg"]}' for err in exc.errors()]
                     ^^^^^^^^^^^^^^^^^^^^
TypeError: sequence item 1: expected str instance, int found

Versions (prefect version output)

Version:             2.19.4
API version:         0.8.4
Python version:      3.11.1
Git commit:          867543a8
Built:               Tue, Jun 4, 2024 3:14 PM
OS/Arch:             linux/x86_64
Profile:             default
Server type:         server

Additional context

This error is raised if the flow is called with invalid parameters.

e.g. if the "name" field is missing from one of the objects being passed.

{
  "params": [
    {"name": "rodrigo"},
    {}
  ]
}

ndamclean avatar Jun 28 '24 16:06 ndamclean