Compatibility Issue with K8s 1.26

[beerus11]

It seems that the latest version of spark-operator is not compatible with K8S 1.26 as there are some APIs used in spark-operator which are deprecated.

[jalkjaer]

I think you need to be more specific. Try to submit the SparkPI example from the getting started guide and report what issues you are seeing (i.e. copy error messages and so on)

[subhranil05]

@jalkjaer apiVersion: sparkoperator.k8s.io/v1beta2 is not supported by k8 1.26 it looks like

{"error": "unable to retrieve the complete list of server APIs: sparkoperator.k8s.io/v1beta2: the server could not find the requested resource"}

Is there any workaround?

[Roberdvs]

I've just tested a fresh Helm install of the spark-operator on 1.26 & 1.27 kind clusters before performing the upgrade on our EKS clusters after seeing this issue and everything looks OK on my end. I was able to run the spark-pi example successfully. Only non-default value on the Helm release was enabling the webhook.

@jalkjaer apiVersion: sparkoperator.k8s.io/v1beta2 is not supported by k8 1.26 it looks like

{"error": "unable to retrieve the complete list of server APIs: sparkoperator.k8s.io/v1beta2: the server could not find the requested resource"}

Is there any workaround?

That error looks like you're missing the CRDs on your cluster when trying to create an SparkApplication resource so make sure you have the CRDs installed and they show when running kubectl api-resources.

[cten]

Based on the docs here I thought the operator installs the CRDs automatically?

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.