kfp-tekton icon indicating copy to clipboard operation
kfp-tekton copied to clipboard

Support runtime configuration in v2

Open diegolovison opened this issue 2 years ago • 1 comments

/kind feature

Description: The current code is

RESOURCE_OP_IMAGE = ":".join(["quay.io/aipipeline/kubectl-wrapper", "latest"])
TEKTON_HOME_RESULT_PATH = "/tekton/home/tep-results/"

# The image to use in basic bash steps such as copying results in multi-step.
TEKTON_BASH_STEP_IMAGE = env.get('TEKTON_BASH_STEP_IMAGE', 'busybox')
TEKTON_COPY_RESULTS_STEP_IMAGE = env.get('TEKTON_COPY_RESULTS_STEP_IMAGE', 'library/bash')
GENERATE_COMPONENT_SPEC_ANNOTATIONS = env.get('GENERATE_COMPONENT_SPEC_ANNOTATIONS', True)

It means that it will be initialized when doing import kfp_tekton

As a developer, I wish to be able to set those variables in the runtime

os.environ["DEFAULT_STORAGE_CLASS"] = ....
os.environ["TEKTON_BASH_STEP_IMAGE"] = ...
os.environ["TEKTON_COPY_RESULTS_STEP_IMAGE"] = ...
os.environ["CONDITION_IMAGE_NAME"] = ....

If you believe that is doable having a fix in 1.5 sounds good. Otherwise, I believe that we should have this in mind for v2.

diegolovison avatar Sep 18 '23 16:09 diegolovison

KFP-Tekton V1 is more of an extension to base Tekton, which we have to do a lot of hacking solution to make it work.

For KFP V2, we don't need these configuration because we are using ml-metadata and Tekton custom tasks to handle all the condition, parameter passing, artifacts etc. However, with the KFP V2 design, none of the KFP pipelines cannot run on pure Tekton because we are trying to get away from pinging k8s API to get all the status.

Tomcli avatar Sep 25 '23 22:09 Tomcli