gvisor icon indicating copy to clipboard operation
gvisor copied to clipboard

Incorrect "mount" in config.json is either silently ignored or crashes runsc

Open stepancheg opened this issue 6 months ago • 3 comments

Description

    "mounts": [
        {
            "destination": "/ddd",
            "source": "/",
            // and if we specify here "type": "bind", runsc crashes
        }
    ]

does not do anything except creating empty directory.

Moreover, specifying "type": "bind" causes runsc to crash.

This is correct version, but this issue is about missing validation, if this configuration is not correct.

"mounts": [
    {
      "destination": "/ddd",
      "source": "/",
      "type": "bind",
      "options": ["rbind", "rw"]
    }
  ]

Steps to reproduce

{
    "process": {
        "args": ["/bin/bash"],
        "terminal": true
    },
    "root": {
        "path": "/"
    },
    "mounts": [
        {
            "destination": "/ddd",
            "source": "/"
        }
    ]
}
$ sudo bazel-bin/runsc/runsc_/runsc run --bundle ~/bu2 cont-x

root@:/# ls -l /ddd 
total 0

root@:/# mount
none on / type overlay (rw)
none on /sys type sysfs (rw,dentry_cache_limit=1000)
none on /dev type tmpfs (rw,mode=0755)
none on /proc type proc (rw,dentry_cache_limit=1000)
none on /dev/pts type devpts (rw)

Log: https://gist.github.com/stepancheg/82cee1f2095405d1c5b790240ad62564

{
    "process": {
        "args": ["/bin/bash"],
        "terminal": true
    },
    "root": {
        "path": "/"
    },
    "mounts": [
        {
            "destination": "/ddd",
            "source": "/",
            "type": "bind"
        }
    ]
}
$ sudo bazel-bin/runsc/runsc_/runsc run --bundle ~/bu2 cont-x

running container: creating container: cannot create sandbox: cannot read client sync file: waiting for sandbox to start: EOF

Log: https://gist.github.com/stepancheg/81c05a3b18909e28716f6cee945c456e

runsc version

runsc version release-20250505.0-31-gd30c58e5b8a4
spec: 1.2.0

uname

Linux w-st 6.6.72+ #1 SMP PREEMPT_DYNAMIC Sat Apr 19 09:07:01 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

repo state (if built from source)

release-20250505.0-31-gd30c58e5b

stepancheg avatar May 16 '25 01:05 stepancheg

When the mount type is not specified, gVisor just ignores the mount: https://github.com/google/gvisor/blob/d30c58e5b8a4ead573d82eba3c8c240eafa21af2/runsc/boot/vfs.go#L929-L931

In this case, runc attempts to mount it and fails with:

WARN[0000] freezer not supported: openat2 /sys/fs/cgroup/user.slice/user-676784.slice/cont/cgroup.freeze: no such file or directory 
WARN[0000] lstat /sys/fs/cgroup/user.slice/user-676784.slice/cont: no such file or directory 
ERRO[0000] runc run failed: unable to start container process: error during container init: error mounting "/" to rootfs at "/ddd": mount /:/ddd (via /proc/self/fd/7): no such device 

After adding "type": "bind" but without the options, the failure happens with gVisor because the gofer fails with:

FATAL ERROR: error setting up FS: mounting {Destination:/ddd Type:bind Source:/ Options:[] UIDMappings:[] GIDMappings:[]}: mount("/", "/proc/fs/root/ddd", 4096) failed: mount("/", "/proc/fs/proc/self/fd/14", "bind", 0x1000, "") failed: invalid argument

This failure is WAI I think.

BTW don't try this reproducer with runc (after adding type=bind), because it can brick your system: https://github.com/opencontainers/runc/issues/4767.

ayushr2 avatar May 16 '25 21:05 ayushr2

When the mount type is not specified, gVisor just ignores the mount:

Thank you for the pointer.

There is some usability issue: if -alsologtostderr is specified, output is too noisy (all "info" is printed there). Otherwise the error is burried in logs. But maybe runsc is meant to be too low level tool to worry about such issues.

After adding "type": "bind" but without the options, the failure happens with gVisor because the gofer fails with: This failure is WAI I think.

End state is probably WAI (I'm not an expert), but diagnostics IMO could be improved:

running container: creating container: cannot create sandbox: cannot read client sync file: waiting for sandbox to start: EOF

Is it hard to forward this error (or whatever error gopher encounters) to the boot process and then to stderr?

stepancheg avatar May 18 '25 04:05 stepancheg

diagnostics IMO could be improved

I agree.

It is because of util.Fatalf() usages in runsc/cmd/gofer.go. It causes gofer to exit and the sandbox fails with:

FATAL ERROR: Error reading mounts file: error unmarshaling mounts: unexpected end of JSON input

Because it is waiting for the gofer to send it some data. And then the sandbox exits here: https://github.com/google/gvisor/blob/d19c3bc9bcdf36fc592d634e9337dd4f84bec02d/runsc/cmd/boot.go#L464-L467

ayushr2 avatar May 18 '25 15:05 ayushr2