spark-operator
spark-operator copied to clipboard
kubeflow
Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.
Hi guys, I am staring a thrift server, and that thrift server starten an exceuter.. But that exceuter wants to cimmunicate with the thrift server on weird ports. Below my...
This PR addresses several issues discussed at length in #216 It allows for additional config files to be mounted into ``SPARK_CONF_DIR`` and adds compatibility to the upstream by changing ``DefaultSparkConfDir``...
Solves https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/issues/1235 Because: https://medium.com/@denisstortisilva/kubernetes-eks-calico-and-custom-admission-webhooks-a2956b49bd0d
We noticed that if we do not include the `sparkSession.stop()` in our Scala app, the executor pod gets stuck in the `Running` state even after the execution is done. Just...
I found mismatched quotes from user-guide.md merged from PR https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/pull/1524
suspend a running spark streaming application We add a new State: suspended. to switch to suspended state we only need to annotate SparkApplication object with a specific annotation. to resume...
Hi all, Sometimes we need to suspend for a short time a running spark. Maybe to manipulate a kafka consumer group, or to prepare some elastic/cassandra/mongo/... entities, or any other...
### Goal: Enable creating prometheus alerts. I'v added here a new template for `PrometheusRule`, and disabled it by default in the values.
### Goal: This PR's goal is to allow adding additional labels to the operator metrics. Currently, the only available label in the operator metrics is `app_type`, but I think it...
