website icon indicating copy to clipboard operation
website copied to clipboard

[Feedback] docs/components/spark-operator/user-guide/gcp.md

Open nrawal47 opened this issue 1 year ago • 10 comments

The specified image does not exist.

nrawal47 avatar Sep 03 '24 18:09 nrawal47

/assign @ChenYi015 @yuchaoran2011 @vara-bonthu @jacobsalway

andreyvelich avatar Sep 04 '24 12:09 andreyvelich

This is referring to gcr.io/ynli-k8s/spark:v2.3.0-gcs right?

jacobsalway avatar Sep 04 '24 23:09 jacobsalway

The GCP related documentation have not been updated for years, I have no idea whether it is still applicable now.

ChenYi015 avatar Sep 05 '24 05:09 ChenYi015

I would suggest to remove this documentation page if that doesn't work now.

andreyvelich avatar Sep 05 '24 13:09 andreyvelich

To be clear, this is the jar that works for us:

jars:
        - "local:///opt/spark/jobs/lib/gcs-connector-3.0.2-shaded.jar"

relevant hadoopConf:

fs.AbstractFileSystem.gs.impl: "com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS"
fs.gs.impl: "com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem" # testing
fs.gs.auth.service.account.enable: "false"
fs.gs.auth.type: "UNAUTHENTICATED"

nrawal47 avatar Sep 06 '24 15:09 nrawal47

@nrawal47 would you like to contribute to this page?

varodrig avatar Jan 12 '25 11:01 varodrig

@nrawal47 would you like to contribute to this page?

Yes but it's been several months so I'm assuming there may be some changes for me to catch up on. I'm actually not using Kubeflow at the moment and working on other things, so my response may be delayed.

nrawal47 avatar Jan 12 '25 16:01 nrawal47

/help

varodrig avatar Jan 13 '25 03:01 varodrig

@varodrig: This request has been marked as needing help from a contributor.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed by commenting with the /remove-help command.

In response to this:

/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

google-oss-prow[bot] avatar Jan 13 '25 03:01 google-oss-prow[bot]

/area spark-operator

varodrig avatar Mar 11 '25 01:03 varodrig

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] avatar Jun 10 '25 00:06 github-actions[bot]

This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.

github-actions[bot] avatar Jul 01 '25 00:07 github-actions[bot]