flytekit
flytekit copied to clipboard
add timestamp transformer
Why are the changes needed?
What changes were proposed in this pull request?
This workflow
@fl.task(container_image=image)
def get_timestamp() -> pd.Timestamp:
return pd.Timestamp("2024-09-29 15:30:45")
@fl.workflow
def timestamp_wf() -> pd.Timestamp:
return get_timestamp()
gives this error:
TypeTransformerFailedError: Failed to convert
outputs of task 'timestamp_wf.get_timestamp' at
position 0.
Failed to convert type <class
'pandas._libs.tslibs.timestamps.Timestamp'> to type
<class 'pandas._libs.tslibs.timestamps.Timestamp'>.
Error Message: Expected value of type <class
'datetime.datetime'> but got '2024-09-29 15:30:45'
of type <class
'pandas._libs.tslibs.timestamps.Timestamp'>.
We need a pandas timestamp type transformer.
How was this patch tested?
Tested locally and remotely. For remote run, just installed flytekit from the da72500c93e1ff1107a24f1e67eb453a443e4d82 commit on the image.
Check all the applicable boxes
- [ ] I updated the documentation accordingly.
- [ ] All new and existing tests passed.
- [ ] All commits are signed-off.