How to properly use ConfigMaps for SparkConf and HadoopConf

[sadatnfs]

Would someone be able to show an end-to-end example of setting up config maps with Spark and Hadoop configurations and applying that to a SparkApplication? I've not been able to properly implement either sparkConfigMap or hadoopConfigMap such that the configs pass through to Spark. Any help or examples would be appreciated, since there's no proper examples available in the repo?

[sadatnfs]

Any help on this please? Sort of been stuck on this for a while.

[powerLambda]

Here is a blocking issue https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/issues/216 for your references.

[marcoboi]

Here is an example of how to pass that map:

  sparkConf:
    spark.ui.prometheus.enabled: "true"

[sadatnfs]

@marcoboi thanks but I was looking for examples for sparkConfDir not sparkConf

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.