fix(backend): add commonEnvs to driver for PVCs
Description of your changes:
This PR adds commonEnvs to the driver container. This will now expose the KFP_POD_NAME and KFP_POD_UID env vars. This PR then updatesInPodName to take advantage of KFP_POD_NAME to accurately get the full pod name without the 63 char truncation issue seen when using hostname (seen in issue #12350)
This supersedes #12351, which attempted to address this issue but was not properly tested. After deploying that change, it was discovered it did not work as expected, due to the ARGO_POD_NAME env var not being exposed in the main container. This PR contains the correct implementation and has been validated in a live cluster.
Testing Evidence:
Using the following pvc hello world file with a long pipeline name:
from kfp import dsl
from kfp import kubernetes
from kfp import compiler
@dsl.component
def producer() -> str:
with open('/data/file.txt', 'w') as file:
file.write('Hello world')
with open('/data/file.txt', 'r') as file:
content = file.read()
print(content)
return content
@dsl.component
def consumer() -> str:
with open('/data/file.txt', 'r') as file:
content = file.read()
print(content)
return content
@dsl.pipeline
def my_pipeline_test_super_long_name_to_check_for_any_possible_errors_maybe():
pvc1 = kubernetes.CreatePVC(
# can also use pvc_name instead of pvc_name_suffix to use a pre-existing PVC
pvc_name_suffix='-my-pvc',
access_modes=['ReadWriteMany'],
size='5Gi',
storage_class_name='standard',
)
# write to the PVC
task1 = producer()
kubernetes.mount_pvc(
task1,
pvc_name=pvc1.outputs['name'],
mount_path='/data',
)
# read to the PVC
task2 = consumer()
kubernetes.mount_pvc(
task2,
pvc_name=pvc1.outputs['name'],
mount_path='/data',
)
task2.after(task1)
delete_pvc1 = kubernetes.DeletePVC(
pvc_name=pvc1.outputs['name']
).after(task2)
# Compile the pipeline
if __name__ == "__main__":
compiler.Compiler().compile(pipeline_func=my_pipeline_test_super_long_name_to_check_for_any_possible_errors_maybe, package_path="kubeflow_pipeline.yaml")
KFP_POD_NAME exposed in main container
Passing Pipeline:
Checklist:
- [X] You have signed off your commits
- [X] The title for your pull request (PR) should follow our title convention. Learn more about the pull request title convention used in this repository.
/lgtm /approve
[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: zazulam
The full list of commands accepted by this bot can be found here.
The pull request process is described here
- ~~OWNERS~~ [zazulam]
Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment