kfp-tekton
kfp-tekton copied to clipboard
Support runtime configuration in v2
/kind feature
Description: The current code is
RESOURCE_OP_IMAGE = ":".join(["quay.io/aipipeline/kubectl-wrapper", "latest"])
TEKTON_HOME_RESULT_PATH = "/tekton/home/tep-results/"
# The image to use in basic bash steps such as copying results in multi-step.
TEKTON_BASH_STEP_IMAGE = env.get('TEKTON_BASH_STEP_IMAGE', 'busybox')
TEKTON_COPY_RESULTS_STEP_IMAGE = env.get('TEKTON_COPY_RESULTS_STEP_IMAGE', 'library/bash')
GENERATE_COMPONENT_SPEC_ANNOTATIONS = env.get('GENERATE_COMPONENT_SPEC_ANNOTATIONS', True)
It means that it will be initialized when doing import kfp_tekton
As a developer, I wish to be able to set those variables in the runtime
os.environ["DEFAULT_STORAGE_CLASS"] = ....
os.environ["TEKTON_BASH_STEP_IMAGE"] = ...
os.environ["TEKTON_COPY_RESULTS_STEP_IMAGE"] = ...
os.environ["CONDITION_IMAGE_NAME"] = ....
If you believe that is doable having a fix in 1.5 sounds good. Otherwise, I believe that we should have this in mind for v2.
KFP-Tekton V1 is more of an extension to base Tekton, which we have to do a lot of hacking solution to make it work.
For KFP V2, we don't need these configuration because we are using ml-metadata and Tekton custom tasks to handle all the condition, parameter passing, artifacts etc. However, with the KFP V2 design, none of the KFP pipelines cannot run on pure Tekton because we are trying to get away from pinging k8s API to get all the status.