Support for buildkit --secret option

[ehengao]

Expected Behaviour

When use docker buildkit to build the function images, there is a feature to use --secret option to create a safer build enviroment, the typical usecase would be when user has a private pypi repository or private gitlab repository which require senstive access information to download in order to install these packages as part of dependency for the function. Currently, the faas-cli build command does not have any option to pass in the secret arguments.

Current Behaviour

Currently, the faas-cli build only have build-arg and build-options that pass sepecific key, value arguements for docker build api. Do not have a way to pass the --secret options and its arguments to the docker build api when buildkit is enabled.

Possible Solution

Add another flags "--buildkit-extra" to pass in the addtional arguments to the docker buildkit. And this options only applicable when user has enabled DOCKER_BUILDKIT=1.

Steps to Reproduce (for bugs)

Context

Your Environment

• FaaS-CLI version ( Full output from: faas-cli version ):0.12.1

• Docker version ( Full output from: docker version ):19.03.8

• Are you using Docker Swarm (FaaS-swarm ) or Kubernetes (FaaS-netes)? faas-swarm

• Operating System and version (e.g. Linux, Windows, MacOS): Linux

• Link to your project or a code example to reproduce issue: N/A

[alexellis]

Thanks for raising your request @ehengao

I'm pinging @LucasRoesler, we're talking about how to support it in the context of #773

[LucasRoesler]

In context of #773, we descussed the idea of having the ability to pass through arbitrary flags

Perhaps with #773 , we can extend

type BuildConfig struct {
	BuildArgs map[string]string `yaml:"build_args"`
}

to have be

type BuildConfig struct {
	BuildArgs map[string]string `yaml:"build_args"`
	BuildFlags []PassThroughFlag `yaml:"build_flags"`
}

type PassThroughFlag struct {
	Name string
	Value string
}

in yaml, for this secret flag it would look like

build_config:
    build_flags:
    - name: secret
      value: id=mysecret,src=mysecret.txt

the corresponding CLI flag would be a repeatable

faas-cli build --build-flag secret=id=mysecret,src=mysecret.txt

For long flags we would probably just split on the first =. I am not sure it would cleanly support short flags though.

These flags can be passed directly to the builder.

[ehengao]

@LucasRoesler, one question, are we supporting putting environment variable into the yaml file?

build_config:
    build_flags:
    - name: secret
      value: id=netrc,src=${HOME}/.netrc

It would be nice to have to resolve the environmnets during runtime, so the yml file is pretty documented for build environment.

[LucasRoesler]

This is already supported

[talhof8]

Any news?

[LucasRoesler]

#842 is related, in wanting to access other flags on the builder. I think https://github.com/openfaas/faas-cli/issues/785#issuecomment-606761803 would allow us to solve both cases, if we decide to add direct support for additional flags to the cli/stack

[goncalo-oliveira]

Using sensitive information on a build process is somewhat common if you're not working with open source. I think most scenarios would be using private package/source registries, although I'm sure there are others. ARGS are not suitable to pass through this sensitive information, as it is retained in the image history, making it vulnerable to anyone with access to the build cache or the built layers. BuildKit support for secrets is a safe alternative, supported since Docker 18.09 (I think) and it would be nice to have a way to integrate with it directly from the faas-cli.

As to how it should be done, I think I would rather see a --secret or --build-secret flag explicitly for this, even if that worked as an alias for the mentioned --build-flags, though, that's just my opinion.

As a workaround, I'm using --shrinkwrap followed by a docker build.

[alexellis]

Given that buildkit's secrets mechanism is not compatible with other container builders such as kaniko, podman and cloud builders, coupling to this solution may exclude a bunch of users.

What would make --shrinkwrap + DOCKER_BUILDKIT=1 docker build easier to use as a solution here?

[goncalo-oliveira]

Given that buildkit's secrets mechanism is not compatible with other container builders such as kaniko, podman and cloud builders, coupling to this solution may exclude a bunch of users.

Makes sense. In that case, I think the best option would be https://github.com/openfaas/faas-cli/issues/785#issuecomment-606761803. That way, the solution is builder-agnostic.

[alexellis]

This feature will be available for OpenFaaS Pro customers.

If you're a customer, please get in touch for instructions on how to use it with our supported templates.

[alexellis]

/lock: resolved