img icon indicating copy to clipboard operation
img copied to clipboard

Building without --privileged under an alpine container in Docker

Open beaugunderson opened this issue 5 years ago • 24 comments

yo, thank you for img! one thing i'm trying to clarify--is it possible to build inside an alpine docker container without using --privileged? the README says this:

Docker image r.j3ss.co/img is configured to be executed as an unprivileged user with UID 1000 and it does not need --privileged since img v0.6.0.

but there's no current release for v0.6.0, and it looks like it was reverted right after it was released...

beaugunderson avatar Apr 16 '19 01:04 beaugunderson

Issue Label Bot is not confident enough to auto-label this issue. See dashboard for more details.

issue-label-bot[bot] avatar Apr 16 '19 01:04 issue-label-bot[bot]

What do you mean by reverted?

AkihiroSuda avatar Apr 16 '19 04:04 AkihiroSuda

0.6.0 was reverted in this commit, yet the README refers to it as the version that works without --privileged

beaugunderson avatar Apr 16 '19 05:04 beaugunderson

the reverted commit seems very old and unrelated.

The planned 0.6.0 will support running without --privileged but not released yet. (You can try master)

AkihiroSuda avatar Apr 16 '19 05:04 AkihiroSuda

@jessfraz could you release 0.6.0?

AkihiroSuda avatar Apr 16 '19 05:04 AkihiroSuda

I've just tried it with master and got this error:

failed to solve: failed to read dockerfile: failed to mount /tmp/buildkit-mount404401060: [{Type:bind Source:/tmp/.local/share/img/runc/native/snapshots/snapshots/2 Options:[rbind ro]}]: operation not permitted

@AkihiroSuda are you sure it will be supported?

AndyBitz avatar May 06 '19 11:05 AndyBitz

are you using gcos? https://github.com/moby/buildkit/issues/879

AkihiroSuda avatar May 06 '19 12:05 AkihiroSuda

@AkihiroSuda No, I've tried it locally on my MacBook with docker run. It only works when I use --privileged

AndyBitz avatar May 06 '19 12:05 AndyBitz

Note that you still need seccomp=unconfined and apparmor=unconfined

AkihiroSuda avatar May 06 '19 12:05 AkihiroSuda

I've tried it with those too, but only the --privileged makes a difference

AndyBitz avatar May 06 '19 12:05 AndyBitz

Whats your kernel?

AkihiroSuda avatar May 06 '19 12:05 AkihiroSuda

4.9.125-linuxkit

AndyBitz avatar May 06 '19 13:05 AndyBitz

@AkihiroSuda my goal is to run this on the amazonlinux base image.

AndyBitz avatar May 06 '19 14:05 AndyBitz

it works for me (4.9.125-linuxkit, Docker 18.09.3, img@013bb211)

$ docker run --rm -it --security-opt seccomp=unconfined --security-opt apparmor=unconfined -v $(pwd):/home/user/src r.j3ss.co/img build -t foo /home/user/src
Building docker.io/library/foo:latest
Setting up the rootfs... this may take a bit.
WARN[0000] Process sandbox is not available, consider unmasking procfs: mount: permission denied (are you root?)

[+] Building 23.0s (6/6) FINISHED
 => [internal] load build definition from Dockerfile                                                                                                                                0.1s
 => => transferring dockerfile: 76B                                                                                                                                                 0.0s
 => [internal] load .dockerignore                                                                                                                                                   0.1s
 => => transferring context: 2B                                                                                                                                                     0.0s
 => [internal] load metadata for docker.io/library/alpine:latest                                                                                                                    3.3s
 => [1/2] FROM docker.io/library/alpine@sha256:28ef97b8686a0b5399129e9b763d5b7e5ff03576aa5580d6f4182a49c5fe1913                                                                     2.2s
 => => resolve docker.io/library/alpine@sha256:28ef97b8686a0b5399129e9b763d5b7e5ff03576aa5580d6f4182a49c5fe1913                                                                     0.0s
 => => sha256:28ef97b8686a0b5399129e9b763d5b7e5ff03576aa5580d6f4182a49c5fe1913 2.36kB / 2.36kB                                                                                      0.0s
 => => sha256:5c40b3c27b9f13c873fefb2139765c56ce97fd50230f1f2d5c91e55dec171907 528B / 528B                                                                                          0.0s
 => => sha256:bdf0201b3a056acc4d6062cc88cd8a4ad5979983bfb640f15a145e09ed985f92 2.76MB / 2.76MB                                                                                      1.8s
 => => sha256:cdf98d1859c1beb33ec70507249d34bacf888d59c24df3204057f9a6c758dddb 1.51kB / 1.51kB                                                                                      0.0s
 => => unpacking docker.io/library/alpine@sha256:28ef97b8686a0b5399129e9b763d5b7e5ff03576aa5580d6f4182a49c5fe1913                                                                   0.3s
 => [2/2] RUN apk add --no-cache gcc                                                                                                                                                5.5s
 => exporting to image                                                                                                                                                             11.8s
 => => exporting layers                                                                                                                                                            11.7s
 => => exporting manifest sha256:184a1f8e1649c7f9276e1130d61ff4db8536239e0f4e408b9632c039d02dd3d6                                                                                   0.0s
 => => exporting config sha256:73863d87f4822166bd36bd32a6f8812439c89c07157d9a558b4be218f16b662a                                                                                     0.0s
 => => naming to docker.io/library/foo:latest                                                                                                                                       0.0s
Successfully built docker.io/library/foo:latest

AkihiroSuda avatar May 06 '19 14:05 AkihiroSuda

I can't seem to get this working with only those 2 security-opt options as well. I'm running an jenkins/jnlp-slave:latest-jdk11 image with the precompiled img 0.5.7 release installed. Running with --privileged resolves it.

rodriguezsergio avatar Aug 15 '19 19:08 rodriguezsergio

Apologies for hijacking the issue, but I am getting the same error as Andy:

Jans-iPhone:~# img build -d -t hello .
DEBU[0000] checking runc                                 disableEmbeddedRunc=false state=/root/.local/share/img
DEBU[0000] runc found                                    commit=56aca5aa50d07548d5db8fd33e9dc562f70f3208 spec=1.0.2 version=1.0.0-rc10+dev
DEBU[0000] using backend: native                        
Building docker.io/library/hello:latest
Setting up the rootfs... this may take a bit.
DEBU[0000] serving grpc connection                      
WARN[0000] Process sandbox is not available, consider unmasking procfs:  
WARN[0000] using host network as the default            
[+] Building 0.0s (0/0)                           
DEBU[0000] reusing ref for local: vt0z8kf22gjwit9y6wqt9s69y 
DEBU[0000] reusing ref for local: lnql54ycndslsamk[+] Building 0.0s (0/2)                           
 => [internal] load build definition from D  0.0s
 => => transferring dockerfile:              0.0s
 => [internal] load .dockerignore            0.0s
 => => transferring context:                 0.0s
DEBU[0000] diffcopy took: 57.044ms                      
DEBU[0000] saved vt0z8kf22gjwit9y6wqt9s69y as loca[+] Building 0.1s (2/2) FINISHED                  
 => [internal] load build definition from D  0.1s 
 => => transferring dockerfile: 2B           0.1s
 => [internal] load .dockerignore            0.1sa
 => => transferring context: 2B              0.1s
Error: failed to solve: failed to solve with frontend dockerfile.v0: failed to read dockerfile: failed to mount /tmp/buildkit-mount078545067: [{Type:bind Source:/root/.local/share/img/runc/native/snapshots/snapshots/1 Options:[rbind ro]}]: invalid argument

The problem is that I am not running this inside a Docker container, but directly on Alpine Linux 3.2.0 i686 kernel. Therefore I cannot just add --privileged. Any advice how to avoid this error when running directly on Alpine? I cannot use Docker.

jankais3r avatar May 05 '20 20:05 jankais3r

We have this problem too. Does anyone have some workaround about this? (to build with img in alpine and unprivileged)

bbkgh avatar Oct 26 '20 07:10 bbkgh

The BuildKit upstream is more actively maintained and has better documentation about running without --privileged https://github.com/moby/buildkit/blob/master/docs/rootless.md#docker

AkihiroSuda avatar Oct 26 '20 07:10 AkihiroSuda

Im running into the same issue when running Gitlab CI with the kubernetes executor. The build job uses an alpine image which fails with:

Error: failed to solve: failed to solve with frontend dockerfile.v0: failed to read dockerfile: failed to mount /tmp/buildkit-mount078545067: [{Type:bind Source:/root/.local/share/img/runc/native/snapshots/snapshots/1 Options:[rbind ro]}]: invalid argument

Adding the pod annotations as suggested in the readme didn't have any effect:

[runners.kubernetes.pod_annotations]
  "container.apparmor.security.beta.kubernetes.io/build" = "unconfined"
  "container.seccomp.security.alpha.kubernetes.io/build" = "unconfined"

Only privileged mode seems to fix this.

baszalmstra avatar Feb 18 '21 14:02 baszalmstra

Im running into the same issue when running Gitlab CI with the kubernetes executor. The build job uses an alpine image which fails with:

Error: failed to solve: failed to solve with frontend dockerfile.v0: failed to read dockerfile: failed to mount /tmp/buildkit-mount078545067: [{Type:bind Source:/root/.local/share/img/runc/native/snapshots/snapshots/1 Options:[rbind ro]}]: invalid argument

Adding the pod annotations as suggested in the readme didn't have any effect:

[runners.kubernetes.pod_annotations]
  "container.apparmor.security.beta.kubernetes.io/build" = "unconfined"
  "container.seccomp.security.alpha.kubernetes.io/build" = "unconfined"

Only privileged mode seems to fix this.

Are you building as root?

oxr463 avatar Feb 18 '21 14:02 oxr463

I dont exactly know what you mean. Im basically running alpine:3.13, I add img with apk add img and run img build:

The gitlab ci job looks like this:

docker:
  image: alpine:3.13
  stage: test
  script:
    - apk add --update --no-cache --quiet img
    - img build --tag ${CI_REGISTRY_IMAGE} .

baszalmstra avatar Feb 18 '21 17:02 baszalmstra

If you are using Google COS kernel, probably you are hitting this https://github.com/moby/buildkit/issues/879

AkihiroSuda avatar Feb 18 '21 23:02 AkihiroSuda

Im running all of this on a DigitalOcean Managed Kubernetes cluster I don't think they are using the same kernel.

baszalmstra avatar Feb 18 '21 23:02 baszalmstra

No resolution yet ?

MrAmbiG avatar Jul 29 '21 12:07 MrAmbiG