feat: Artifact GC
Fixes #8476
Note: may want to hold off on merging this until Dillen Padhiar's PR is merged which executes the logic in the Garbage Collection Pod (argoexec artifact delete). Although, his first draft code is currently contained within this branch, which is functional, if we choose to merge this first.
I have tested this under the following conditions:
- different combinations of specifying ArtifactGC (Strategy, ServiceAccount, PodMetadata) on the Workflow level and Artifact level
- ArtifactGC failure
- Workflows based on StoredTemplates
- Workflows whose steps reference templates in StoredTemplates
- Workflows that use Steps and Workflows that use DAG
The following is not being implemented as part of this MVP and can be done later:
- OnWorkflowSuccess,OnWorkflowFailure (with deletion)
- Validation: Jesse suggested we reject the Workflow during validation if
ArtifactGCis configured along with a non-supported storage engine (for now probably anything besides S3). - Creating multiple ArtifactGCTasks in the case that we start to exceed 1MB
- handling recreation of the GC Pod in case it's evicted or deleted
I think you should descope the OnWorkflowFailure and OnWorkflowSuccess, validate the use case and if valid, follow up with another PR.
I removed those strategies and can create a new Issue. "OnWorkflowSuccessOrDeletion" and "OnWorkflowFailureOrDeletion" may make sense.