pipelines icon indicating copy to clipboard operation
pipelines copied to clipboard

[feature] Support for location arguments for `create_custom_training_job_from_component`

Open faisalron opened this issue 1 year ago • 4 comments

Feature Area

Google Cloud Pipeline Component for custom job. Function name: create_custom_training_job_from_component

What feature would you like to see?

Currently, components created by create_custom_training_job_from_component will be set to the default 'us-central1' location and there is no way to change that.

What is the use case or pain point?

Sometimes we need to execute all of components of a pipeline on a specific region due to the network latency or just because the regulations. Unable to specify the location blocks this requirement.

Is there a workaround currently?

Currently, I need to define my component as container component and wrap it up with CustomTrainingJobOp to be able to set the location. https://google-cloud-pipeline-components.readthedocs.io/en/google-cloud-pipeline-components-2.8.0/api/v1/custom_job.html#v1.custom_job.CustomTrainingJobOp


Love this idea? Give it a 👍.

faisalron avatar Feb 22 '24 08:02 faisalron

@faisalron, I can run successfully the single component in this pipeline in us-east1. Would you mind sharing your pipeline code?

from kfp import dsl
from google_cloud_pipeline_components.v1 import custom_job

@dsl.component
def print_foo():
    print('foo')

print_foo = custom_job.create_custom_training_job_from_component(print_foo)

@dsl.pipeline
def my_pipeline():
    print_foo(location='us-east1')

connor-mccarthy avatar Mar 13 '24 00:03 connor-mccarthy

@connor-mccarthy Thanks for checking in. I didn't know that we can set location using that argument.

print_foo(location="us-east-1") # I didn't know it generates location argument.

Do we have any documentation on that?

faisalron avatar Mar 13 '24 07:03 faisalron

@faisalron, for some reason project and location are omitted from the docs. I will look into that.

connor-mccarthy avatar Mar 13 '24 16:03 connor-mccarthy

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] avatar May 13 '24 07:05 github-actions[bot]

This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.

github-actions[bot] avatar Jun 03 '24 07:06 github-actions[bot]