Container creation fails for images with too many layers

[imre-kerr-sb1]

Description

When running a container with too many layers (71 or more in my testing), docker fails with the following message:

docker: Error response from daemon: failed to create shim task: OCI runtime create failed: error in the container spec: failed to request rootfs cloning from sysbox-mgr: failed to invoke ReqCloneRootfs via grpc: rpc error: code = Unknown desc = failed to mount clone for container 99e8e3a5d6a5: failed to set up bottom ovfs mount: failed to mount overlayfs on /var/lib/sysbox/rootfs/99e8e3a5d6a5ca47390327b57b581fbfeb76dcbb21c8ef1de703a1a0e1c51e73/bottom/merged: no such file or directory: unknown. ERRO[0000] error waiting for container: context canceled

Steps to reproduce

1. Create a Dockerfile with the following content:

From busybox:latest

RUN mktemp
RUN mktemp
# ... Repeat 100 times

2. docker build . -t too-many-layers
3. docker run --runtime=sysbox-runc too-many-layers

System info

• Sysbox version: 0.5.2
• Docker version: 20.10.23, build 7155243
• Linux version: 5.15.0-1026-aws
• Distribution: Ubuntu 22.04.1 LTS

[ctalledo]

Thanks @imre-kerr-sb1 for reporting the error; have never seen this one before.

Curious how you hit it? Did you come across it with a real image?

[imre-kerr-sb1]

Curious how you hit it? Did you come across it with a real image?

Yes. The image in question was based on gitpod/workspace-full (54 layers), with 26 layers of our own stuff on top. Of course this is easily fixable by being more economical with layers or even squashing them. But I still figured I should report it as it's obviously a bug.