gvisor icon indicating copy to clipboard operation
gvisor copied to clipboard

Warning when using an unsupported version of Linux

Open ianlewis opened this issue 5 years ago • 10 comments

gVisor fails in unintuitive ways when run on an older version of Linux. We should print a warning or simply fail if he version of Linux used is less than the currently supported version.

ianlewis avatar May 31 '19 11:05 ianlewis

Currently runsc fails with an error like the following because the kernel is missing the memfd_create syscall which was added in Linux 3.17

OCI runtime create failed: unable to retrieve OCI runtime error (open /run/containerd/io.containerd.runtime.v1.linux/moby/2779a57fb4979db21996621118dd72bf58842937f0de04613184e8b68f2659bc/log.json: no such file or directory): /usr/local/bin/runsc did not terminate sucessfully: unknown.

See #100

ianlewis avatar May 31 '19 11:05 ianlewis

In theory, we do have a warning: https://github.com/google/gvisor/blob/master/pkg/sentry/memutil/memutil_unsafe.go#L34

I know the plumbing all the way up to the Docker CLI worked when that was added, but it must have regressed since then.

prattmic avatar May 31 '19 16:05 prattmic

I think it broke when a synchronization point was added to sandbox.New(). It will be fixed with: https://gvisor-review.git.corp.google.com/c/gvisor/+/18260

fvoznika avatar Jun 03 '19 17:06 fvoznika

Hi all,

I traced this issue from here My machine said xxx/log.json: no such file or directory, like this: image

However, It supports memfd_create syscall. I have no idea what's happened?

And my gVisor version is: "Version release-20210906.0"

25077667 avatar Sep 26 '21 02:09 25077667

I believe you're running into #5865 (you need to enable cgroup v1 controller). If you run sudo runsc do echo 123 you should get a more decent error message.

fvoznika avatar Sep 29 '21 01:09 fvoznika

This is something that also occurs when running inside unprivileged LXC containers (Kernel 5.10) , but I haven't figured out why that is yet. There shouldn't be any reason why that syscall is blocked there...

I0411 20:24:11.652735  1453859 main.go:219] ***************************
I0411 20:24:11.652910  1453859 main.go:220] Args: [/usr/local/bin/runsc --debug --debug-log=/tmp/runsc-debug.log --strace --log-packets --root /var/run/docker/runtime-runc/moby --log /run/containerd/io.containerd.runtime.v2.task/moby/c5d4227bdbf1811fb7e9d8a4619a773e9ed65e87a857662549c86c1b272ab3d6/log.json --log-format json delete --force c5d4227bdbf1811fb7e9d8a4619a773e9ed65e87a857662549c86c1b272ab3d6]
I0411 20:24:11.652948  1453859 main.go:221] Version release-20220228.0
I0411 20:24:11.652957  1453859 main.go:222] GOOS: linux
I0411 20:24:11.652979  1453859 main.go:223] GOARCH: amd64
I0411 20:24:11.652989  1453859 main.go:224] PID: 1453859
I0411 20:24:11.652999  1453859 main.go:225] UID: 0, GID: 0
I0411 20:24:11.653007  1453859 main.go:226] Configuration:
I0411 20:24:11.653015  1453859 main.go:227] 		RootDir: /var/run/docker/runtime-runc/moby
I0411 20:24:11.653024  1453859 main.go:228] 		Platform: ptrace
I0411 20:24:11.653038  1453859 main.go:229] 		FileAccess: exclusive, overlay: false
I0411 20:24:11.653062  1453859 main.go:230] 		Network: sandbox, logging: true
I0411 20:24:11.653072  1453859 main.go:231] 		Strace: true, max size: 1024, syscalls: 
I0411 20:24:11.653081  1453859 main.go:232] 		VFS2 enabled: true, LISAFS: false
I0411 20:24:11.653099  1453859 main.go:233] 		Debug: true
I0411 20:24:11.653107  1453859 main.go:234] ***************************
D0411 20:24:11.653156  1453859 state_file.go:52] Load container, rootDir: "/var/run/docker/runtime-runc/moby", id: {SandboxID: ContainerID:c5d4227bdbf1811fb7e9d8a4619a773e9ed65e87a857662549c86c1b272ab3d6}, opts: {Exact:false SkipCheck:false}
W0411 20:24:11.653342  1453859 delete.go:74] couldn't find container "c5d4227bdbf1811fb7e9d8a4619a773e9ed65e87a857662549c86c1b272ab3d6": file does not exist
I0411 20:24:11.653413  1453859 main.go:250] Exiting with status: 0

With this docker daemon.json:

{
    "runtimes": {
        "runsc": {
            "path": "/usr/local/bin/runsc"
        },
        "runsc-debug": {
            "path": "/usr/local/bin/runsc",
            "runtimeArgs": [
                "--debug",
                "--debug-log=/tmp/runsc-debug.log",
                "--strace",
                "--log-packets"
            ]
        }
    },
    "storage-driver": "fuse-overlayfs"
}

outis151 avatar Apr 11 '22 20:04 outis151

@outis151 I'm not totally sure that's the same problem. runsc needs root in order to run for a few reasons. You can try running it in an unprivileged context (e.g. as an non-root user) using the --rootless flag and see if that works for you though there are some caveats like you can't use host networking and a few others I think.

ianlewis avatar Apr 11 '22 23:04 ianlewis

runsc is actually running as root user inside the user namespace of the lxc container which is unprivileged. The result is the same running it with the --rootless flag. I am suspecting that this may be a cgroup problem

outis151 avatar Apr 12 '22 09:04 outis151

@outis151 yes, by root I mean a privileged root user. --rootless will skip using cgroups for the reason you are alluding to. While it still could be cgroup related, if you're getting the same result when using it then I'm guessing that your problem lies elsewhere. https://github.com/google/gvisor/blob/bf86207401cab99d859b25acd4911038608f0d33/runsc/container/container.go#L1279-L1290

aside: I was actually wrong. You can't only use host networking or no networking with --rootless https://github.com/google/gvisor/blob/4e99f17178ed39b55adfda3966d51a9860995211/runsc/cmd/run.go#L70-L73

ianlewis avatar Apr 12 '22 10:04 ianlewis

Thanks for the suggestions. Will continue trying to find what is causing this

outis151 avatar Apr 13 '22 15:04 outis151