pipelines icon indicating copy to clipboard operation
pipelines copied to clipboard

[sdk] Missing Alternative for set_ttl_seconds_after_finished in KFP v2

Open nasenthi opened this issue 1 year ago • 6 comments

Environment

  • KFP version: KFP v2
  • KFP SDK version: 2.5.0

Steps to reproduce

  1. Create and run any pipeline in KFP v2.
  2. Observe that the pods created by the run are not automatically deleted after the pipeline execution is complete.

In KFP v1, there was a function kfp.dsl.PipelineConf().set_ttl_seconds_after_finished(seconds) that allowed us to delete pods after a specified time. However, in KFP v2, this function has been deprecated. When attempting to use this deprecated function, we encounter errors as documented in the following issues:

Issue - https://github.com/kubeflow/pipelines/issues/10715 Issue - https://github.com/kubeflow/pipelines/issues/9803 Is there any alternative method available in KFP v2 to delete the pods after they have finished executing?

Expected result

There should be a method in KFP v2 SDK to specify the TTLStrategy inside argo workflows to have them cleaned up.

Materials and Reference


Impacted by this bug? Give it a 👍.

nasenthi avatar Jun 13 '24 06:06 nasenthi

I'm working on this

gregsheremeta avatar Jun 18 '24 11:06 gregsheremeta

/assign @gregsheremeta

hbelmiro avatar Jun 18 '24 11:06 hbelmiro

We are currently migrating all our pipelines to V2.7.0 and experienced the same problem. It was also useful to have a way to add secrets at a pipeline level and image pull secrets. Thanks @gregsheremeta 👍

papagala avatar Jun 20 '24 08:06 papagala

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] avatar Aug 20 '24 07:08 github-actions[bot]

It was also useful to have a way to add secrets at a pipeline level and image pull secrets

Extracted PipelineConfig to #11112 -- please :+1: there :)

gregsheremeta avatar Aug 20 '24 10:08 gregsheremeta

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] avatar Oct 20 '24 07:10 github-actions[bot]

/lifecycle frozen

juliusvonkohout avatar Oct 21 '24 09:10 juliusvonkohout

/assign @rimolive

gregsheremeta avatar Oct 21 '24 12:10 gregsheremeta