caddy-builder broken on hosts where setcap is not supported

[ptxmac]

I'm running docker on my Synology NAS.

Synology have their own variant of linux, and they provide the docker installation which is a bit... wonky.

Among other things, setcap is not supported

So when trying to use the caddy:2.5.4-builder image docker fails with:

2023/04/11 21:32:04 [INFO] Setting capabilities (requires admin privileges): [setcap cap_net_bind_service=+ep ./caddy]
Failed to set capabilities on file './caddy': Not supported
2023/04/11 21:32:04 [ERROR] failed to setcap on the binary: exit status 1
The command '/bin/sh -c xcaddy build' returned a non-zero code: 1

Adding ENV XCADDY_SETCAP 0 to my Dockerfile solves the problem, but I had to look through commits and source code to discover that. It would be helpful if it was mentioned on https://hub.docker.com/_/caddy

Even better if support for setcap could be auto-detected, but I have no knowledge about that

[francislavoie]

:grimacing:

I didn't even know that could fail if the command existed. That's annoying.

I know nothing about Synology. Would you be able to find out why it doesn't work? I have no idea how to determine if setcap would fail.

Anyways, instructions for xcaddy are documented here https://github.com/caddyserver/xcaddy#environment-variables and we do link to there from https://hub.docker.com/_/caddy so I think that's sufficient.

I think using XCADDY_SETCAP=0 is the correct fix for this case, since we don't know currently why it fails or how to check ahead of time.

[ptxmac]

I think it's because synology still use aufs as the storage driver, and from what I can find setcap isn't supported on aufs: https://github.com/moby/moby/issues/5650

[hairyhenderson]

Is this a build-only error? i.e. could you build the image elsewhere and pull it onto the NAS?

[ptxmac]

Hmm, that might work, though it doesn't fit well in my current setup to have this built in a different host. For now just setting XCADDY_SETCAP=0 fixes my immediate problem

[hairyhenderson]

Ok, thanks for the feedback!

[dmgawel]

This might be also problematic for running official images on services with more restrictive permissions, that doesn't allow cap_net_bind_service. Example would be Render.com.

I struggled with deploying Caddy on Render.com (caddy: operation not permitted), and thing that helped was removing all set caps:

FROM caddy:latest

RUN setcap -r /usr/bin/caddy

[polarathene]

@dmgawel this was covered here: https://github.com/containerd/nerdctl/issues/2439#issuecomment-1865495690

• setcap is used to grant the capability for a non-root user, provided the capability is in the containers bounding set when starting the caddy process.
• It's common as a default capability granted to the container, but dropping all caps as a default security measure would cause the error you experienced.
• setcap is still relevant to grant caddy the ability to raise the capability for it's own usage, but it shouldn't be relying on setcap to enforce that, but instead do so at runtime. Sometimes a capability is enforced via setcap even when you wouldn't use the functionality that'd require it, so blocking via kernel enforcement is considered poor practice.

Personally I don't have much issue with the container running a process as root. For those that are more security conscious about such they should be using rootless containers which maps the internal container root UID to a non-root UID on the system.

Main perk of non-root in the container is all capabilities are stripped and in the event that the container had mount access to important content on the host filesystem owned by root, it'd be prevented from writing.