Philipp Dallig

I think downgrading to CentOS 6 is not an option. Checkout the [End of Lifetime page](https://wiki.centos.org/About/Product). We should rather prefer an upgrade to centos 8. I think the ssh part...

You are right, running SSH daemon in a docker container is a very bad practice. I'm not sure that's necessary. After reading the [documentation spark-on-yarn-mode](https://zeppelin.apache.org/docs/0.9.0-SNAPSHOT/setup/deployment/spark_cluster_mode.html#spark-on-yarn-mode), it seems that only one...

I think most Spark configuration values can be included in the environment variable `SPARK_SUBMIT_OPTIONS`. Please note that the K8sProcess works for all Zeppelin interpreters.

HI @zlosim As you can see in the following lines, SPARK_SUBMIT_OPTIONS is concatenated with the spark configurations specifically for Kubernetes. https://github.com/apache/zeppelin/blob/8995ff227a44760b7dbb005a85bee4ac7772e1f2/zeppelin-plugins/launcher/k8s-standard/src/main/java/org/apache/zeppelin/interpreter/launcher/K8sRemoteInterpreterProcess.java#L323-L325 Your configuration values like `spark.driver.extraClassPath` and `spark.driver.extraLibraryPath` should take...

I understand your motivation. I just don't like the list of Spark configuration at this point, because it can become very long. Spark has very many configuration options and we...

I would like to test this beforehand in my local K8s cluster, but I am still on vacation right now. Please have a little patience.

Hi @zlosim Sorry for the late reply. I tried to test your branch. Currently, I am getting a startup error because I have already fully transitioned my environment to Spark-3.3....

"Cluster.local" as you mentioned is not always correct. I would prefer either a domain value that can be configured or add documentation that you need to configure your cluster DNS...

> Do you want me to create a config entry ? Something like k8s_cluster_search_domain Yes, please. This is then a possible breaking change (for 1%). Please update your PR description....

Yes, I can write a bit of documentation. Where should this be added? Is `examples/dashboard.json` the right place for the dashboard.json?