[backend] Some environment variables missing in 1.8.5 manifest examples/samples for GCP

[cjidboon94]

Two things seem to be missing from the GCP env kustomize (for version 1.8.5 at least) mysql-secret is not being generated nor set in some kind of params.env Nor is defaultPipelineRoot set in the params to a reasonable variable. I would expect it to be generated from the bucketName variable that is to be set in params.env

mysql-secret I figured out by having a look at the Kustomization in https://github.com/kubeflow/pipelines/blob/1.8.5/manifests/kustomize/sample

defaultPipelineRoot I only figured out after getting the following error message:

F0123 11:48:24.180376      21 main.go:50] Failed to execute component: failed to upload output artifact "local_path" to remote storage URI "minio://mlpipeline/v2/artifacts/pipeline/raw-pipeline/30a0867f-4b5f-4fe6-baa0-8b60cd0c5e84/decode-hevc/local_path": uploadFile(): unable to complete copying "/minio/mlpipeline/v2/artifacts/pipeline/raw-pipeline/30a0867f-4b5f-4fe6-baa0-8b60cd0c5e84/decode-hevc/local_path/[redacted].json" to remote storage "decode-hevc/local_path/[redacted].json": failed to close Writer for bucket: blob (key "decode-hevc/local_path/[redacted].json") (code=Unknown): AccessDenied: Access Denied.
	status code: 403, request id: 17ACF783C1A1A7D2, host id: 
time="2024-01-23T11:48:24.181Z" level=error msg="cannot save artifact /tmp/outputs/local_path/data" argo=true error="stat /tmp/outputs/local_path/data: no such file or directory"
Error: exit status 1

By adding a sensible value defaultPipelineRoot to my params.env and retried the above pipeline, the issue was resolved.

Environment

• How did you deploy Kubeflow Pipelines (KFP)? I installed from manifests using the following instructions: https://www.kubeflow.org/docs/components/pipelines/v1/installation/standalone-deployment/#deploy-on-gcp-with-cloud-sql-and-google-cloud-storage
• KFP version: 1.8.5
• KFP SDK version: Irrelevant as this is about installation via manifests

Steps to reproduce

Install according to above instructions.

Expected result

mysql-secret and defaultPipelineRoot to be set in some ConfigMap/Secret (or asked somewhere to be set in either the documentation or have some reasonable defaults)

Materials and Reference

---

Impacted by this bug? Give it a 👍.

[zijianjoy]

Hello @cjidboon94 , would you like to upgrade KFP version from 1.8.5 to 2.0.5 or above? This 1.8.5 version is very old, we can check whether it is successful after upgrade.

[cjidboon94]

Hey @zijianjoy I hope to pretty soon. My pipelines require some refactoring before that's possible so for now will remain on 1.8.5. Will let you know once I update and if I run into the same issue.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[github-actions[bot]]

This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.