Clean up pods via TTL Seconds After Finished by configuring HELM deployment instead of individual Jobs.

[dangal95]

What's the use case?

The Kubernetes Executor creates a new Pod everytime a job runs. The pod takes a while to get deleted when the job is finished, unless you specify a tag per job in the following way:

tags={
            "dagster-k8s/config": {
                "job_spec_config": {
                    "ttl_seconds_after_finished": TIME_IN_SECONDS
                } 
            }
        },

The feature I am requesting would be to create a configuration that is part of the Dagster deployment, and not specific jobs, so that all Job pods can be cleaned up quicker.

Ideas of implementation

Add this functionality as a configuration in the values.yaml file used to deploy Dagster on Kubernetes.

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.