[BUG] `PanderaTransformer::to_python_value()` seems to be returning an incorrect type

[danieldanciu]

The return type of the to_python_value of the pandera type transformer seems incorrect. Currently, the returned type is pandas.core.frame.DataFrame, but the declared return type is pandera.typing.DataFrame. I suspect this is because instantiating pandera.DataFrameSchema returns a pandas.core.frame.DataFrame, rather than what one would normally expect, namely a pandera.typing.DataFrame.

I will propose an MR to fix this shortly.

[welcome[bot]]

Thank you for opening your first issue here! 🛠

[runllm[bot]]

The issue arises because PanderaTransformer::to_python_value() returns a pandas.DataFrame instead of a pandera.typing.DataFrame. This discrepancy is due to the instantiation of pandera.DataFrameSchema, which returns a pandas.DataFrame.

Would you like more detail on how to address this issue?

Sources:

• flytekit-pandera/schema.py
• PanderaTransformer.to_python_value
• flytekitplugins.pandera.schema

---

This is an AI-generated response and your feedback is appreciated! Please leave a 👍 if this is helpful and 👎 if it is not.

Mention @runllm to ask follow-up questions. See runllm.com for more!

[wayner0628]

Hi @danieldanciu,

The pandera.typing.DataFrame is a generic type for pandas.DataFrame. You can refer to it here. Therefore, it should be fine for to_python_value() to return a pandas.core.frame.DataFrame type.

cc: @pingsutw