spark-operator
spark-operator copied to clipboard
kubeflow
Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.
It seems tha the way that: https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/blob/master/pkg/controller/sparkapplication/sparkui.go manages ingressPath is peculiar to nginx ingress controller For others like traefik the grouping in case of (/|$)(.*) has to be dome differently...
Would someone be able to show an end-to-end example of setting up config maps with Spark and Hadoop configurations and applying that to a SparkApplication? I've not been able to...
Hi, we have been running Spark applications via the Spark operator for a while now. We recently updated from K82 v1.25 to v1.27 (AWS EKS, to be precise) and on...
Add PVC permission to Operator role, so that Spark Operator able to create Driver pod with PVC by follow spark configs: ``` spark.kubernetes.driver.volumes.persistentVolumeClaim.checkpointpvc.options.claimName: "driver" spark.kubernetes.driver.volumes.persistentVolumeClaim.checkpointpvc.options.storageClass: "gp2" spark.kubernetes.driver.volumes.persistentVolumeClaim.checkpointpvc.options.sizeLimit: "50Gi" spark.kubernetes.driver.volumes.persistentVolumeClaim.checkpointpvc.mount.path: "/data"...
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.0.0-20220315160706-3147a52a75dd to 0.17.0. Commits See full diff in compare view [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) You can trigger a rebase of this PR by commenting `@dependabot rebase`. [//]: #...
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot] avatar"){kind=avatar}
It has a lot of usage, but I'm unable to tell how active it is. A lot of the go modules are starting to trigger vulnerability scanners, so I'm curious...
Is it possible to set ingressClassName in an ingress object? I have several ingresses in the cluster (including nginx only for spark) and I would like to be able to...
Hi masters, I'm trying out preStop hook supported by latest spark-operator, and I noticed the preStop hook failed if I run some time-consuming task or simply sleep for a while...
While trying to run spark operator with Spark 3.5.0, I noticed the following errors: ``` Message: Forbidden!Configured service account doesn't have access. Service account may have been revoked. services "spark-pi-test-run-1-72fa8a8c0fd5639e-driver-svc"...
Hi all, I seem to be having some issues with the getting a spark application up and running: hittig issues like this: ``` 21/06/04 07:42:53 INFO SparkKubernetesClientFactory: Auto-configuring K8S client...
