spark-operator icon indicating copy to clipboard operation
spark-operator copied to clipboard

Volumes not mounted inside initcontainer

Open grantatspothero opened this issue 4 years ago • 1 comments

If a volumeMount is configured inside an initcontainer of the driver/executor, the volume is not actually mounted into the pod unless the volume is also mounted inside the driver/executor respectively.

Example:

kind: SparkApplication
spec:
  ...
  volumes:
    - name: git-secret
      secret:
        secretName: git-creds
        defaultMode: 0400
  driver:
    ...
    initContainers:
      - name: "git-sync"
        volumeMounts:
         - name: git-secret
           mountPath: "/etc/git-secret"
    # If the volume is not mounted inside the driver, it will not be mounted inside the initcontainer
    #volumeMounts:
    #  - name: git-secret
    #    mountPath: "/etc/git-secret"

The root cause is the spark operator only mounts volumes to pods based on the driver/executor volume mounts and does not consider the volume mounts of the init containers. See here: https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/blob/7731b7eb8d1cb74eb171f60dbf411ddcdfd9ed81/pkg/webhook/patch.go#L134-L143

grantatspothero avatar Jan 27 '21 20:01 grantatspothero