Secrets as SparkCluster environment variables

[windmark]

Description:

I'm using the SparkCluster to connect to sensitive systems and I'm trying to pass a secret reference as the environment variable value. Is this a supported feature and I'm simply doing something wrong in my configuration? Although it seems like the CRD schema definition doesn't include it. If not supported, is there any limitations preventing this feature in general?

Steps to reproduce:

Creating a SparkCluster with the following settings

apiVersion: radanalytics.io/v1
kind: SparkCluster
metadata:
  name: spark-cluster
spec:
  env:
    - name: USER
      value: test
    - name: PASSWORD
      valueFrom:
        secretKeyRef:
          name: password-secret-name
          key: password

results in an environment variable without a value

apiVersion: radanalytics.io/v1
kind: Pod
metadata:
  name: spark-cluster-X
spec:
  containers:
    - resources:
      env:
        - name: USER
          value: test
        - name: PASSWORD

I expect the env value reference to the password-secret-name to be in place also for the cluster pods.