Mount type/options are not optional in my experience.

[oreillymj]

In the CDI spec, both mount type and mount options are marked as optional. My experience is that the mount fails without an access option set. The error is also pretty confusing.

Using dev0 from this CDI file fails

cdiVersion: 0.6.0
kind: vendor.com/device
devices:
    - name: dev0
      containerEdits:
        env:
            - SOME_VAR=dev0
            - FOO=injected
        mounts:
            - hostPath: /usr/lib/firmware
              containerPath: /usr/lib/firmware
              type: ""
              options: []

with the following error

Warning Failed 12s kubelet Error: failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/usr/lib/firmware" to rootfs at "/usr/lib/firmware": mount src=/usr/lib/firmware, dst=/usr/lib/firmware, dstFd=/proc/thread-self/fd/8: no such device

If the mounts have a type and options, then the same pod is created sucessfully

        mounts:
            - hostPath: /usr/lib/firmware
              containerPath: /usr/lib/firmware
              type: bind
              options:
                - rbind
                - ro

To replicate I have created a skeleton device plugin here , which advertises 8 device to kublet and writes a matching CDI file with or without the mount type/options. with the device plugin running, you can test it by scheduling this pod

apiVersion: v1
kind: Pod
metadata:
  name: busybox-bad-mount
spec:
  containers:
  - name: busybox
    image: busybox
    command: ["sleep", "infinity"]
    resources:
      limits:
        vendor.com/device: 1  # Request 1 device
  restartPolicy: Never
  nodeSelector: {}

[elezar]

The optionality of the fields is taken directly from the OCI runtime spec with the relevant sections being specified:

• https://github.com/opencontainers/runtime-spec/blob/main/config.md#mounts
• https://github.com/opencontainers/runtime-spec/blob/main/config.md#posix-platform-mounts

[oreillymj]

If I look at that OCI spec, it's hard to deduce that the options are not required. Lots of options are marked as MUST support.

Bottom line, if I look at the OCI JSON generated by containerd, the only item being mounted into the pod without a type or options is the item from my CDI file, and runc fails with an error.

So no matter what that spec says, the reality is the the field is required.

sudo crictl inspect 535f79d00a8e11ec3ec7f2c1933296ea75a2f363fc111731c259726d7a23cd5d

.....

        {
          "destination": "/etc/resolv.conf",
          "options": [
            "rbind",
            "rprivate",
            "rw"
          ],
          "source": "/var/lib/containerd/io.containerd.grpc.v1.cri/sandboxes/ac1118c671b81c9929359d7d2362d4df6a788f8925a0a17952eb633274e961a3/resolv.conf",
          "type": "bind"
        },
        {
          "destination": "/dev/shm",
          "options": [
            "rbind",
            "rprivate",
            "rw"
          ],
          "source": "/run/containerd/io.containerd.grpc.v1.cri/sandboxes/ac1118c671b81c9929359d7d2362d4df6a788f8925a0a17952eb633274e961a3/shm",
          "type": "bind"
        },
        {
          "destination": "/sys/fs/cgroup",
          "options": [
            "nosuid",
            "noexec",
            "nodev",
            "relatime",
            "ro"
          ],
          "source": "cgroup",
          "type": "cgroup"
        },
        {
          "destination": "/usr/lib/firmware",
          "source": "/usr/lib/firmware"
        },
        {
          "destination": "/var/run/secrets/kubernetes.io/serviceaccount",
          "options": [
            "rbind",
            "rprivate",
            "ro"
          ],
          "source": "/var/lib/kubelet/pods/7ac1e611-6ce6-40a5-8bf4-554425b0084e/volumes/kubernetes.io~projected/kube-api-access-rhgwk",
          "type": "bind"
        }
      ],

[klihub]

@elezar @oreillymj I suspect that either option or type is optional but not both. IOW, you need to have at least one of them, but it depends on the specifics of your mount which is the mandatory one... If this is true, then we could add a check that at least one of them is present.

[klihub]

@elezar @oreillymj I suspect that either option or type is optional but not both. IOW, you need to have at least one of them, but it depends on the specifics of your mount which is the mandatory one... If this is true, then we could add a check that at least one of them is present.

Although I have to say that so far I failed to find any mount which wouldn't require a type...

[elezar]

Since the spec is interpreted by the low-level runtime, the behaviour depends on the implementation there. For example, in runc: https://github.com/opencontainers/runc/blob/f023e1c222f0b2c2584544a44d7b734cd6c311a5/libcontainer/specconv/spec_linux.go#L652-L661

the Type field is ignored for bind mounts.

[oreillymj]

Type may be ignored by runc, but from memory, if I didn't have rbind (or maybe bind) in the Options for a bind mount, then runc failed to setup the pod.

[elezar]

Type may be ignored by runc, but from memory, if I didn't have rbind (or maybe bind) in the Options for a bind mount, then runc failed to setup the pod.

Yes, that would be the expected behavior since from the runc perspective, type is required iff options does not include bind or rbind.

In terms of closing out this issue, would clearing out documentation on the spec be sufficient from your perspective? Or should we raise an error explicitly when checking validity?

[oreillymj]

I'm building/writing the CDI file programmatically without importing this package for validation, so the labelling of items in the spec is important. But how you label mount types/options is a tricky one. The attributes may not be required, but they are also not optional. For me, I need the documented spec to be accurate. For others, who rely on this package for validation then all types, options would need to be aligned with what the various actually support.