pipelines
pipelines copied to clipboard
[sdk] gcp secret depreciated
Environment
- KFP version: 1.7
- KFP SDK version: 1.8.22
- All dependencies version:
Steps to reproduce
In, kfp v1, I use the line below to pass the service account to the pipeline pod
dsl.get_pipeline_conf().add_op_transformer(gcp.use_gcp_secret('user-gcp-sa'))
However, by using kfp.v2.dsl, I get
module 'kfp.v2.dsl' has no attribute 'get_pipeline_conf'
Using the new method below works
@dsl.pipeline(
name='GCP access in pipeline',
)
def cred_pipeline():
from kfp.gcp import use_gcp_secret
container_op = read_gcs().apply(use_gcp_secret('user-gcp-sa'))
But I see that the gcp file has been moved to kfp.depreciated in v2, do we have any alternative to use a service account key in v2?
Expected result
Materials and Reference
Impacted by this bug? Give it a 👍.
@N214 For running Pipelines on GCP, we are recommending to use Vertex SDK, where you can specify the service account there.
cc @connor-mccarthy
@N214 For running Pipelines on GCP, we are recommending to use Vertex SDK, where you can specify the service account there.
Ah interesting ! Thank you
Just to explain a little bit more my use case here, I have on-prem hardware that I wish to use to train my models, but some datasets are store in GCS or Bigquery, so I just need a service account to be passed to the pipeline pod to have access to GCP. The pipelines should run on prem, not on GCP.
Thank you
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.