kfp-tekton
kfp-tekton copied to clipboard
kubeflow
optimize step-copy-results-artifacts script to minimize etcd space
/kind feature
Description: step-copy-results-artifacts script has too many variable that get stringify during compile time. We need to rewrite it in a way that it takes a variable as the inputs for source and destination path instead of raw string
Additional information: [Miscellaneous information that will assist in solving the issue.]
cc @ScrapCodes @yhwang not sure is one of you have time to take a look and reduce the script size in the generated yaml? There was few changes made into the big data passing and now the generated step-copy-results-artifacts script is getting way too big.
Mainly the past few PRs are from different contributors are duplicating the src and dst python variables into raw strings. So we want to rewrite it in a way that to use Tekton variables to reduce the final yaml size.