User namespace remapping doesn't work with multi-stage builds

[shaneog]

User namespace remapping doesn't work when you use multi-stage builds such as with the below Dockerfile.

Is this possible to fix, or am I stuck with using user namespace remapping OR multi-stage builds?

References: https://github.com/moby/moby/issues/34645 https://github.com/buildkite/agent/issues/407

Example Repo: https://github.com/shaneog/bk-issue-392

Error:

ERROR: Service 'app' failed to build: failed to copy files: failed to copy file: Container ID 500000 cannot be mapped to a host ID

Dockerfile:

FROM debian:stretch as build
RUN touch x && chown 1:1 x

FROM debian:stretch
RUN touch y && chown 1:1 y
COPY --from=build x ./

Using Buildkite stack v2.3.5 The stack was created with all default values, except for the agent token, and enabling "User namespace remapping"

[lox]

Thanks @shaneog, this is a tricky one! 🤔

[lox]

A possible workaround is to chown -R root:root ... in the --from image and then restore the ownership in the target image.

From the moby issue, might that be viable in your case @shaneog?

[shaneog]

I've disabled user namespace remapping for now, and I'm using the permission scripts to fix any file ownership issues.

[lox]

We've brought back EnableDockerUserNamespaceRemap in #410. You can use this in https://s3.amazonaws.com/buildkite-aws-stack/master/aws-stack.json and we'll be cutting a 3.1.0 release soon.

[lox]

I'm going to re-open this one, as it's still an issue when user namespace remapping is turned on.

[pda]

Looks like it's still an issue in the latest buildkite/elastic-ci-stack-for-aws with default configuration?

[pda]

(for some reason upgrading the stack from v3.2.1 to v4.2.0 has triggered this issue for us, even though v3.2.0 seems to be where EnableDockerUserNamespaceRemap was enabled by default)

[pda]

((ah — we had EnableDockerUserNamespaceRemap set to false in the old stack, and true in the new stack))

[lox]

Yup, still an issue @pda.

[jufemaiz]

👀