spark-operator
spark-operator copied to clipboard
kubeflow
Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.
They do not run with the following error: > 2021/12/01 18:32:44 failed to set up framework: setup test environment failed: setup operator failed: failed to create operator service account: failed...
Hi, I have a use case were we have a set of Spark Batch Applications with a Scheduled Interval, I was able to schedule the batch application by using the...
We're installing the spark op to multiple namespaces. Our trick for now is to installCRDs only for the first install. The problem is that a second install breaks the first...
Hello, we set up `spark-on-k8s-operator` with UI enabled. We provide dynamic DNS and automatic `letsencrypt` certificates in our cluster so it is possible to create any url on HTTPS automatically...
Hello All, I have made changes to values.yaml uiService: ## - Enable UI service creation for Spark application enable: true ## - Ingress URL format. ## Requires the UI service...
Image used : `gcr.io/spark-operator/spark:v3.1.1` kubernetes client jar: `kubernetes-client-4.12.0.jar` We are getting this issue intermittently Relevant Logs: ``` io.fabric8.kubernetes.client.KubernetesClientException: too old resource version at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager$1.onMessage(WatchConnectionManager.java:258) at okhttp3.internal.ws.RealWebSocket.onReadMessage(RealWebSocket.java:323) at okhttp3.internal.ws.WebSocketReader.readMessageFrame(WebSocketReader.java:219) at okhttp3.internal.ws.WebSocketReader.processNextFrame(WebSocketReader.java:105)...
In relation to issue https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/issues/1281, this PR adds client mode support. When client mode is chosen, upon application submission, a driver is pod is created instead of the spark submit...
Hello there, I want to import [spark-on-k8s-operator] to rosbag analysis system. The Ros can only be installed in ubuntu. Is there anyone can make a ubuntu(16 is the best version...
When pulling the Helm Chart from https://googlecloudplatform.github.io/spark-on-k8s-operator and installing the latest (=spark-operator-1.1.12) I run into `flag provided but not defined: -webhook-timeout` Most likely because of [this](https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/commit/6939cbd887adda19984ed5f75fd49dcbfc7c09b4) commit. When I install...
Hello everyone! I have some issues using Volume for sparkApplication. I''m describing the use of persistentVolumeClaim in the yaml file for SparkApplication as described in the documentation: https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/blob/master/docs/user-guide.md#mounting-volumes However, the...
