Mounting a ConfigMap storing Spark Configuration Files doesn't valid

[wenfang6]

l use sparkConfigMap to mout spark-defaults.conf, in this file l set this: spark.sql.extensions org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions spark.sql.catalog.spark_catalog org.apache.iceberg.spark.SparkSessionCatalog spark.sql.catalog.spark_catalog.type hive spark.sql.catalog.hive_test org.apache.iceberg.spark.SparkCatalog spark.sql.catalog.hive_test.type hive spark.sql.catalog.hive_test.uri thrift://ip:port but these parameters are not valid。 and l see the image，the spark-defaults.conf is exists in the directory （/etc/spark/conf），so l am confused why these parameters are not valid? if l missed something...

[ringtail]

Is there any other log?

[wenfang6]

just not found my iceberg table. now i just use spec.sparkConf to set these conf is valid. use spec.sparkConfigMap is not valid. but actually l want to use spec.sparkConfigMap to set conf

[tutututu-1]

This is because the k8s driver spark-submit with --properties-file=spark.properties, you can see this information in the driver's log. But when spark-submit has --propertiesfile parameter, spark will not consider spark-default.conf anymore. You can fix this by modifying the SparkSubmitArguments.mergeDefaultSparkProperties method in the spark source code.