[Bug] [Cloud-K8S] spark-pi-fffb478d5856a8b3-driver run in default namespace

[hugh018]

Search before asking

• [X] I had searched in the issues and found no similar issues.

What happened

Cloud -> k8s ，Submit spark task，configure namespace bigdata-dev, But spark-pi-fffb478d5856a8b3-driver run in default namespace. bigdata-dev cloud-k8s-126-t47nh 0/1 Completed 0 5m32s default spark-pi-fffb478d5856a8b3-driver 0/1 Completed 0 5m17s

What you expected to happen

Spark-pi run in configure namespace, Not default namespace

How to reproduce

Spark 3.4.1 dolphinscheduler 3.2.0 ["/opt/spark/bin/spark-submit"] ["--master","k8s://https://xx.xx.xx.xx:6443/","--deploy-mode", "cluster","--name", "spark-pi","--class", "org.apache.spark.examples.SparkPi","--conf","spark.executor.instances=1","--conf","spark.kubernetes.authenticate.driver.serviceAccountName=spark-service-account","--conf","spark.kubernetes.container.image=apache/spark:3.4.1","local:///opt/spark/examples/jars/spark-examples_2.12-3.4.1.jar"]

Anything else

No response

Version

3.2.x

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[pegasas]

https://spark.apache.org/docs/3.5.0/running-on-kubernetes.html#customized-kubernetes-schedulers-for-spark-on-kubernetes maybe you add spark.kubernetes.namespace to your script will work.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

[github-actions[bot]]

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.