spark-operator
spark-operator copied to clipboard
kubeflow
Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.
I created a spark application with a custom docker image. I installed the spark operator in the same namespace with version [ghcr.io/googlecloudplatform/spark-operator:v1beta2-1.3.3-3.1.1](http://ghcr.io/googlecloudplatform/spark-operator:v1beta2-1.3.3-3.1.1). Here is how I start my spark session:...
Removes `k8s.io/kubernetes` dependency by replacing with appropriate subcomponents. (https://github.com/kubernetes/kubernetes/issues/79384#issuecomment-505725449) > Copied from the referenced golang issue thread: > > k8s.io/kubernetes is not primarily intended to be consumed as a module....
Hi all, I'm trying to start a simple spark-pi application using custom image with spark 3.3.0. The operator logs showed the spark-pi-driver is running. The driver pod logs showed however...
Once https://github.com/apache/spark/pull/21669/files is pulled into the mainline spark, there will be kerberos support for HDFS clusters that use kerberos for authentication. I wasn't able to find anything about the approach...
is there any readiness/liveness probes for the spark-operator itself. we are using the spark-operator for driver and executor pod recovery in case of failure. as part of the organization policy...
Add new features: When a user submits a job to k8s using the native spark-submit, the spark-operator also senses the creation of the driver pod and creates the sparkApplication CR...
Dear Maintainers, I would like to ask a question a challenge regarding the usage of the spark-k8s operator: We would like to trigger the execution of a SparkApplication right after...
Fix position of end statements in helm chart webhook templates. 😄 Typo of position of theses end statement causes the following error: `Error: UPGRADE FAILED: error validating "": error validating...
Spark Thrift Server is a daemon server that can execute spark sql through JDBC/ODBC connector. It can be usefully used in hive's execution engine and BI tool that supports JDBC/ODBC....
