[FEATURE]: Allow build time secrets

[markgeejw]

Feature Request

Background / Motivation

I am trying to use Garden to replace my workflow with kubernetes. However, a big part of my original workflow with local docker involves using docker build --secret to pass in build time secrets. After much effort around trying to hack this with the current system, it doesn't seem possible to do this securely.

I'm using the buildkit builder.

PS: I've tried to hack this by using the extraFlags to try to use the <() syntax to create a tempfile with the token, but it seems like the kube api client sanitizes commands to not run any nested shells.

What should the user be able to do?

The user should be able to pass in their build time secrets to their build, securely.

Why do they want to do this? What problem does it solve?

The only way to do this without this feature is to use buildArgs, which exposes the secret in all subsequent layers as well as build logs. We COULD do multi-stage builds, but that is un-ideal if the original Dockerfile doesn't already do this.

Suggested Implementation(s)

There are two options to do this, based on how docker buildkit has exposed the secrets interface - environment and files:

1. Allow users to mount additional secret volumes. We can then use extraFlags to do --secret id=TOKEN,src=<mounted>
2. Allow users to pass in an environment variable to the exec. We can then use extraFlags to do --secret id=TOKEN,env=<ENV>

How important is this feature for you/your team?

🥀 Crucial, Garden is unusable for us without it

[stefreak]

This is now partially possible when using Buildkit or local-docker since #5712

There is no first-class support in Garden yet; That definitely makes sense and we will address that at some point.

Here is a work-around solution until then:

kind: Build
type: exec
name: write-secrets
spec:
  command:
    - "sh"
    - "-c"
    - |
        mkdir -p .secrets;
        echo "$SECRET_FROM_ENV" > .secrets/secret-from-env
  env:
    SECRET_FROM_ENV: ${local.env.SECRET_FROM_ENV}
variables:
  dockerExtraFlags:
    - '--secret'
    - 'id=secret-from-env,src=./.secrets/secret-from-env'

---
kind: Build
type: container
name: container-using-secrets
dependencies:
  - build.write-secrets
copyFrom:
  - build: write-secrets
    sourcePath: .secrets
spec:
  extraFlags:
    - $concat: ${actions.build.write-secrets.var.dockerExtraFlags}

With this solution, keep in mind that the .secrets folder will be part of your build context. This means you need to be careful not to accidentally include the secrets into the image using COPY or ADD statements. With first-class support for Docker secrets in Garden we can avoid that problem.

[ArchieY393]

write-secrets creates a .secrets folder and keeps it in .garden/build temporarily to pass it on to build action container-using-secrets. Is this solution recommended as secrets get stored in the .garden/build directory? I tried to add .secrets to .gardenignore but still it preserves in the build directory.

[stefreak]

Thank you for raising awareness @ArchieY393.

We're working on first-class secret support. Unfortunately kaniko won't support that due to lack of support upstream (https://github.com/GoogleContainerTools/kaniko/issues/3028), but local build mode, BuildKit-in-cluster-building and Garden Cloud Builder will be supported.

[stefreak]

@ArchieY393 @markgeejw @soren121 In the latest Bonsai-edge release (and with the next stable release) container Build actions support a new field spec.secrets:

kind: Build
type: container
name: test
spec:
  # ...
  secrets:
    mytoken: verysecret

These secrets are then mountable in the Dockerfile like so:

  RUN --mount=type=secret,id=mytoken TOKEN=$(cat /run/secrets/mytoken) ...

Docs: https://docs.garden.io/v/edge-release/reference/action-types/build/container#spec.secrets

Feel free to give it a try by updating to edge-bonsai using the command garden self-update edge-bonsai, and let us know what you think.