pipelines icon indicating copy to clipboard operation
pipelines copied to clipboard

fix(backend): add commonEnvs to driver for PVCs

Open JerT33 opened this issue 1 month ago • 1 comments

Description of your changes:

This PR adds commonEnvs to the driver container. This will now expose the KFP_POD_NAME and KFP_POD_UID env vars. This PR then updatesInPodName to take advantage of KFP_POD_NAME to accurately get the full pod name without the 63 char truncation issue seen when using hostname (seen in issue #12350)

This supersedes #12351, which attempted to address this issue but was not properly tested. After deploying that change, it was discovered it did not work as expected, due to the ARGO_POD_NAME env var not being exposed in the main container. This PR contains the correct implementation and has been validated in a live cluster.

Testing Evidence:

Using the following pvc hello world file with a long pipeline name:

from kfp import dsl
from kfp import kubernetes
from kfp import compiler

@dsl.component
def producer() -> str:
    with open('/data/file.txt', 'w') as file:
        file.write('Hello world')
    with open('/data/file.txt', 'r') as file:
        content = file.read()
    print(content)
    return content

@dsl.component
def consumer() -> str:
    with open('/data/file.txt', 'r') as file:
        content = file.read()
    print(content)
    return content


@dsl.pipeline
def my_pipeline_test_super_long_name_to_check_for_any_possible_errors_maybe():
    pvc1 = kubernetes.CreatePVC(
        # can also use pvc_name instead of pvc_name_suffix to use a pre-existing PVC
        pvc_name_suffix='-my-pvc',
        access_modes=['ReadWriteMany'],
        size='5Gi',
        storage_class_name='standard',
    )

    # write to the PVC
    task1 = producer()
    kubernetes.mount_pvc(
        task1,
        pvc_name=pvc1.outputs['name'],
        mount_path='/data',
    )

    # read to the PVC
    task2 = consumer()
    kubernetes.mount_pvc(
        task2,
        pvc_name=pvc1.outputs['name'],
        mount_path='/data',
    )
    task2.after(task1)

    delete_pvc1 = kubernetes.DeletePVC(
        pvc_name=pvc1.outputs['name']
    ).after(task2)

# Compile the pipeline
if __name__ == "__main__":
    compiler.Compiler().compile(pipeline_func=my_pipeline_test_super_long_name_to_check_for_any_possible_errors_maybe, package_path="kubeflow_pipeline.yaml")

KFP_POD_NAME exposed in main container Screenshot 2025-11-30 at 10 11 44 PM

Passing Pipeline: Screenshot 2025-11-29 at 6 10 09 PM


Checklist:

JerT33 avatar Dec 01 '25 03:12 JerT33

/lgtm /approve

zazulam avatar Dec 16 '25 13:12 zazulam

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: zazulam

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

google-oss-prow[bot] avatar Dec 16 '25 13:12 google-oss-prow[bot]