spark-operator
spark-operator copied to clipboard
Volumes not mounted inside initcontainer
If a volumeMount
is configured inside an initcontainer of the driver/executor, the volume is not actually mounted into the pod unless the volume is also mounted inside the driver/executor respectively.
Example:
kind: SparkApplication
spec:
...
volumes:
- name: git-secret
secret:
secretName: git-creds
defaultMode: 0400
driver:
...
initContainers:
- name: "git-sync"
volumeMounts:
- name: git-secret
mountPath: "/etc/git-secret"
# If the volume is not mounted inside the driver, it will not be mounted inside the initcontainer
#volumeMounts:
# - name: git-secret
# mountPath: "/etc/git-secret"
The root cause is the spark operator only mounts volumes to pods based on the driver/executor volume mounts and does not consider the volume mounts of the init containers. See here: https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/blob/7731b7eb8d1cb74eb171f60dbf411ddcdfd9ed81/pkg/webhook/patch.go#L134-L143