Retroactive/historical `TTLStrategy` and `activeDeadlineSeconds`

[alexec]

Summary

Currently you can set activeDeadlineSeconds and ttlStrategy on a per workflow basis, or as a default. But neither apply retroactively.

Motivation

So we can clean-up historical workflows automatically, reducing operating costs.

Proposal

For new workflows we can apply them automatically when we set-up defaults.

For historical workflows, we can't do activeDeadlineSeconds. This is applied to pods when they are created.

For historical workflows, we already have an TTL informer that will delete them. What we could do is when that informer starts, if they are missing the value, and we think they should have it, we patch them.

---

Message from the maintainers:

If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

[simster7]

activeDeadlineSeconds only applies to Running workflows... how would it be used to clean up historical workflows?

[alexec]

For historical workflows, we can't do activeDeadlineSeconds.