flyte icon indicating copy to clipboard operation
flyte copied to clipboard

[Core feature] Support Pydantic Models for input/output task/workflow types

Open cosmicBboy opened this issue 2 years ago • 0 comments

Motivation: Why do you think this is important?

pydantic is a widely-used python data validation library. Similar to data classes, it offers a flexible way for users to create custom types that map field names to a variety of value types.

As a pydantic user, I can use my Model definitions in my Flyte workflows so that I don't have to re-implement my custom-defined types in a format that Flyte understands, like data classes.

Goal: What should the final outcome look like, ideally?

I should be able to use pydantic.BaseModel subclasses in my flyte tasks and workflows

from datetime import datetime
from typing import List, Optional
from pydantic import BaseModel


class User(BaseModel):
    id: int
    name = 'John Doe'
    signup_ts: Optional[datetime] = None
    friends: List[int] = []


from flytekit import task


@task
def my_task(user: User) -> User:
    ...

Describe alternatives you've considered

Create some sort of utility function that converts BaseModels to dataclasses.

Propose: Link/Inline OR Additional context

No response

Are you sure this issue hasn't been raised already?

  • [X] Yes

Have you read the Code of Conduct?

  • [X] Yes

cosmicBboy avatar Jul 10 '22 15:07 cosmicBboy