Support dnsPolicy and dnsConfigs for operator and webhook in Helm chart.

[messerzen]

Hello. I'm trying to use the spark-on-k8s-operator using a different dnsConfig and dnsPolicy due to some restrictions and limitations of k8s cluster that I'm using.

Something like this in values.yaml of helm chart:

dnsConfig:
          nameservers:
          - XXX.XX.X.XXX
dnsPolicy: ClusterFirstWithHostNet

This is necessary because, in my case, if I don't set dnsPolicy: ClusterFirstWithHostNet for the pods in my cluster, it's not solving the dns kubernetes.default.svc used to initialize the webhook. The image below shows the error if I don't set dnsPolicy: ClusterFirstWithHostNet (it happens for both init-webhook and clean-up-webhook)

I need to use the webhook activated in order to some customizations in SparkApplication take effect (like also, configure dnsConfig and dnsPolicy) for driver and executors, otherwise, it will not take effect.

Thanks!

[leticiasss]

I'm having the same issue, any updates on this?

[messerzen]

Hi @leticiasss. In my case, the k8s cluster network was fixed and it started to work ( the webhook ). I think one possible workaround for it is to generate the yaml mafinests files from the helm chart, edit the manifests network configs and deploy it manually.

[leticiasss]

Hi @messerzen. Thank you for updated. Initially the installation unsuccessful, but did it again, then worked as expected.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[github-actions[bot]]

This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.