Backlog: add Pydantic type validation to the task decorator

[marvin-robot]

Opened from the Prefect Public Slack Community

andreas.ntonas: HI! I would like to ask if parameters passed to a task in Prefect 2.0 are also validated with pydantic or is this the case only in the flow level. (For example if I have provided pydantic annotations/field types or a pydantic model to a downstream task will prefect still check the data the upstream task passes to the downstream task in runtime?)

anna: For now, we only support pydantic type validation in flows, not in tasks. There are some reasons for that: • This validation is mainly needed so that when you create your run, e.g. via UI or API call, you can pass custom parameter values and this way, they will be validated/type-checked • this level of validation is not that needed for tasks since you can write arbitrary Python within your flow and tasks to perform validation Having said that, I will add your request to a backlog and perhaps sometime later this year, we can consider adding pydantic to tasks as well

<@ULVA73B9P> open "Backlog: add Pydantic type validation to the task decorator"

Original thread can be found here.

[zanieb]

Interesting. As noted, we include this for flows so we can JSON parameters from the API but task runs do not track their parameters this way as they are always passed inline. It seems like it'd be weird to add this to tasks as an option and have it off by default but on by default for flows? Hm.

[andreas-ntonas]

I think it will be nice to have it as an option. It removes the need to write custom data validation inside a task or to write a separate intermediate task to validate data. It can simplify some flows where there is a strict need to check the internal data passed between tasks.

Maybe setting it off by default for tasks but providing the capability for such stricter internal validation is the way to go

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. To keep this issue open remove stale label or comment.

[github-actions[bot]]

This issue was closed because it has been stale for 14 days with no activity. If this issue is important or you have more to add feel free to re-open it.