RangerSparkExtension - ConfigMaps not deleted on Kubernetes

[maron546]

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

Search before asking

• [X] I have searched in the issues and found no similar issues.

Describe the bug

On Kubernetes, after each query, even if all the pods are deleted successfully, a ConfigMap still remains without related resources. The ConfigMap is named:

spark-exec-...-conf-map

and contains the following configuration files:

ranger-spark-audit.xml
ranger-spark-security.xml

Affects Version(s)

1.7.1

Kyuubi Server Log Output

Not applicable

Kyuubi Engine Log Output

Not applicable

Kyuubi Server Configurations

kyuubi.authentication=KERBEROS
kyuubi.frontend.connection.url.use.hostname=false
kyuubi.frontend.protocols=THRIFT_BINARY
kyuubi.ha.addresses=xxx
kyuubi.ha.namespace=kyuubi
kyuubi.kinit.principal=xxx
kyuubi.kinit.keytab=xxx
kyuubi.kubernetes.authenticate.caCertFile=xxx
kyuubi.kubernetes.authenticate.oauthTokenFile=xxx
kyuubi.kubernetes.namespace=xxx
kyuubi.metadata.store.jdbc.database.schema.init=false
kyuubi.metrics.enabled=true
spark.master=xxx
spark.driver.memory=xxx
spark.executor.memory=xxx
spark.dynamicAllocation.enabled=true
spark.dynamicAllocation.initialExecutors=1
spark.dynamicAllocation.maxExecutors=xxx
spark.dynamicAllocation.shuffleTracking.enabled=true
spark.sql.adaptive.enabled=true
spark.kubernetes.container.image=xxx
spark.kubernetes.container.image.pullPolicy=xxx
spark.kubernetes.driver.annotation.app.kubernetes.io/component=kyuubi
spark.kubernetes.driver.request.cores=xxx
spark.kubernetes.driver.limit.cores=xxx
spark.kubernetes.driverEnv.HADOOP_CONF_DIR=/opt/kyuubi/conf
spark.kubernetes.executor.annotation.app.kubernetes.io/component=kyuubi
spark.kubernetes.executor.request.cores=xxx
spark.kubernetes.executor.limit.cores=xxx
spark.kubernetes.executor.secrets.kyuubi-configs=/opt/kyuubi/conf
spark.executorEnv.HADOOP_CONF_DIR=/opt/kyuubi/conf
spark.kubernetes.namespace=xxx
spark.kerberos.access.hadoopFileSystems=xxx
spark.jars=/opt/ranger-plugin/libext/*.jar
spark.sql.extensions=org.apache.kyuubi.plugin.spark.authz.ranger.RangerSparkExtension

Kyuubi Engine Configurations

Not applicable

Additional context

No response

Are you willing to submit PR?

• [ ] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix.
• [ ] No. I cannot submit a PR at this time.