Notify success or failure of pipelines to Slack or Email

[sfujiwara]

System information

• TFX Version (you are using): 1.0.0
• Environment in which you plan to use the feature (e.g., Local (Linux/MacOS/Windows), Interactive Notebook, Google Cloud, etc..): MacOS
• Are you willing to contribute it (Yes/No): Yes

Describe the feature and the current behavior/state.

I want to notify the success or failure of TFX Pipelines to Slack or Email.

Will this change the current API? How?

In Kubeflow, ExitHandler is executed after all tasks finish. https://stackoverflow.com/questions/57508382/kubeflow-pipeline-termination-notificaiton

In Airflow, we can use Email configuration or callbacks: https://airflow.apache.org/docs/apache-airflow/stable/howto/email-config.html https://airflow.apache.org/docs/apache-airflow-providers-dingding/stable/operators.html#sending-messages-from-a-task-callback

Who will benefit with this feature?

I think it is important to use TFX on production.

Do you have a workaround or are completely blocked by this? :

To notify success, we can use custom component, but I could not find the way to notify failure.

Name of your Organization (Optional)

Any Other info.

[jiyongjung0]

Hi, sorry for the late follow up. We added a concept called "Exit Handler" to Vertex Pipelines and Kubeflow Pipelines. https://github.com/tensorflow/tfx/blob/master/RELEASE.md#version-140 Please take a look.

[singhniraj08]

@sfujiwara,

As mentioned in above comment, we have added a concept of exit_handler to annotate the component for post actions of a pipeline. Kindly let us know if this helps your use case.

Thank you!

[sfujiwara]

Thanks, I will try it.