pipelines icon indicating copy to clipboard operation
pipelines copied to clipboard

[sdk] Cannot get real job_id by using PIPELINE_JOB_ID_PLACEHOLDER in component

Open aoyifei opened this issue 2 years ago • 1 comments

Environment

  • KFP version: 2.0.5
  • KFP SDK version: 2.5.0
  • All dependencies version: kfp 2.5.0 kfp-kubernetes 1.0.0 kfp-pipeline-spec 0.2.2 kfp-server-api 2.0.5

Steps to reproduce

just following this script run a test case https://github.com/kubeflow/pipelines/blob/master/sdk/python/test_data/pipelines/pipeline_with_placeholders.py ` from kfp import compiler from kfp import dsl from kfp.dsl import component

@component def print_op(msg: str, value: str): print(msg, value)

@dsl.pipeline(name='pipeline-with-placeholders') def my_pipeline():

print_op(
    msg='job name:',
    value=dsl.PIPELINE_JOB_NAME_PLACEHOLDER,
)
print_op(
    msg='job resource name:',
    value=dsl.PIPELINE_JOB_RESOURCE_NAME_PLACEHOLDER,
)
print_op(
    msg='job id:',
    value=dsl.PIPELINE_JOB_ID_PLACEHOLDER,
)
print_op(
    msg='task name:',
    value=dsl.PIPELINE_TASK_NAME_PLACEHOLDER,
)
print_op(
    msg='task id:',
    value=dsl.PIPELINE_TASK_ID_PLACEHOLDER,
)

if name == 'main': compiler.Compiler().compile( pipeline_func=my_pipeline, package_path=file.replace('.py', '.yaml'))`

Expected result

I could get real id or name of pipeline or task metadata, but not just meaningless string like {{$.pipeline_job_uuid}}

image

Materials and Reference

Also, the use example of placeholders is confused, I want to create folder for every run job, and save the output file in it, should I using placeholder get the run_id like in kfp v1? I don't think when we using a placeholder in component or pipeline could only get something like this {{$.pipeline_job_uuid}} image


Impacted by this bug? Give it a 👍.

aoyifei avatar Feb 07 '24 07:02 aoyifei

@aoyifei unfortunately, these placeholders are currently not supported by KFP runtime.

Tracking issue awaits community contributors: https://github.com/kubeflow/pipelines/issues/6155

chensun avatar Feb 16 '24 00:02 chensun

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] avatar Apr 16 '24 07:04 github-actions[bot]

This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.

github-actions[bot] avatar May 08 '24 07:05 github-actions[bot]

/reopen

SumanthAluri-ARAIC avatar Jul 17 '24 06:07 SumanthAluri-ARAIC

@SumanthAluri-ARAIC: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

google-oss-prow[bot] avatar Jul 17 '24 06:07 google-oss-prow[bot]