Support docker build-time secret

[Lou1415926]

Currently, to pass sensitive information to docker build while keeping it out of manifest, folks can use "shell environment variable interpretation":

image:
  build:
    args:
      TOKEN: ${TOKEN}

However, even though the value of TOKEN is kept out of the manifest, it is still visible in docker history. This might not be a sufficient degree of security for some users.

It'd be great if Copilot supports docker build-time secret: docker build --secret. Note that this requires docker buildkit, and users should be responsible for mounting the secret themselves in their Dockerfiles.

Related: https://github.com/aws/copilot-cli/issues/4224

Edit(2023/11/15) I'd like to append a second ask from this feature is to pass SSM/secretsmanager secret as docker build args. Related https://github.com/aws/copilot-cli/issues/2794#issuecomment-1691295150.

[lokeshsipani]

Hi @iamhopaul123, will this feature enable us to pull build time args (secrets) from AWS Systems Manager. I mean, will it support having different values of args depending upon the environment?

image:
  build:
    dockerfile: Dockerfile
    args:
      REACT_APP_SECRET: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/SECRET

[Lou1415926]

@lokeshsipani I'm afraid not from AWS Systems Manager, unfortunately 😞

I mean, will it support having different values of args depending upon the environment?

This however is a different story and is totally doable already, using environment overrides

image:
  build:
    dockerfile: Dockerfile
    args:
      REACT_APP_SECRET: ${MY_REACT_APP_SECRET}

environments:
  test:
    image:
      build:
        args:
          REACT_APP_SECRET: ${TEST_REACT_APP_SECRET}
  prod:
    image:
      build:
        args:
          REACT_APP_SECRET: ${PROD_REACT_APP_SECRET}

When you run copilot svc deploy --env test, the image will be built using ${TEST_REACT_APP_SECRET}. Similarly, when running copilot svc deploy --env prod, the image will be built using ${PROD_REACT_APP_SECRET}.

[lokeshsipani]

@Lou1415926 Can we have fetching build time args from AWS Systems Manager as a feature request? Eventually, we want to have all our build and run time secrets in one place to avoid inconsistency.

[Lou1415926]

@lokeshsipani Understood your concern 👍🏼 The main concern from Copilot's side would be security; once that's figured out and deemed mitigable, I don't see larger technical obstacles that prevent Copilot from supporting it.

By the way, if you are using CI/CD, you should be able to do the following:

1. In buildspec, us env section to inject secrets from AWS Systems Manager to the build environment, as env vars
2. Reference the env vars using the ${} syntax in your manifest

[nibblesnbits]

Is there any word on this support? I use private packages in my build and cannot authenticate with the repository without feeding those secrets to docker following this guide.

[ryanc-scalegrowth]

Also looking for this feature. Getting secrets from our AWS parameter store or Secrets Manager services and directly inserting them into the docker build phase would be great.

[ShanikaEdiriweera]

@lokeshsipani I'm afraid not from AWS Systems Manager, unfortunately 😞

I mean, will it support having different values of args depending upon the environment?

This however is a different story and is totally doable already, using environment overrides

image:
  build:
    dockerfile: Dockerfile
    args:
      REACT_APP_SECRET: ${MY_REACT_APP_SECRET}

environments:
  test:
    image:
      build:
        args:
          REACT_APP_SECRET: ${TEST_REACT_APP_SECRET}
  prod:
    image:
      build:
        args:
          REACT_APP_SECRET: ${PROD_REACT_APP_SECRET}

When you run copilot svc deploy --env test, the image will be built using ${TEST_REACT_APP_SECRET}. Similarly, when running copilot svc deploy --env prod, the image will be built using ${PROD_REACT_APP_SECRET}.

@Lou1415926 will this work in pipeline as well. I assume the image is built only once in that case?

[Lou1415926]

Hey @ShanikaEdiriweera! If you are asking about environment overrides, then yes - the environments field, would work the same away in pipeline as in copilot svc deploy!

I assume the image is built only once in that case?

Technically, this depends on your buildspec.yml - although Copilot generates a buildspec.yml for you when your run copilot pipeline init, you are free to modify that buildspec file however you want, so the actual behavior might change based on how you've modified the buildspec.

But based on the default buildspec that Copilot generates - yes, the image is indeed built once per environment. If your manifest contains test and prod stage, than the build stage will build one image for test, and another image for prod - one build per env, there are two envs involved in the pipeline, hence two builds in total.

This can be sped up by the cache_from field: if both images are similar, then the second build of the similar image will be largely sped up.

[ShanikaEdiriweera]

Hey @ShanikaEdiriweera! If you are asking about environment overrides, then yes - the environments field, would work the same away in pipeline as in copilot svc deploy!

I assume the image is built only once in that case?

Technically, this depends on your buildspec.yml - although Copilot generates a buildspec.yml for you when your run copilot pipeline init, you are free to modify that buildspec file however you want, so the actual behavior might change based on how you've modified the buildspec.

But based on the default buildspec that Copilot generates - yes, the image is indeed built once per environment. If your manifest contains test and prod stage, than the build stage will build one image for test, and another image for prod - one build per env, there are two envs involved in the pipeline, hence two builds in total.

This can be sped up by the cache_from field: if both images are similar, then the second build of the similar image will be largely sped up.

Cool. thanks alot! I will try it out. One question, can we use ${COPILOT_ENVIRONMENT_NAME} as build arg? Will the manifest substitute the value as correct environment or is the value substituted only at build time?

[Lou1415926]

can we use ${COPILOT_ENVIRONMENT_NAME} as build arg? Will the manifest substitute the value as correct environment or is the value substituted only at build time?

Yes, you should be able to do that. Copilot will substitute ${COPILOT_ENVIRONMENT_NAME} with the correct environment name before build time - i.e. by the time when it starts building the images, it already knows that wherever ${COPILOT_ENVIRONMENT_NAME} was should be that environment name.

[ShanikaEdiriweera]

can we use ${COPILOT_ENVIRONMENT_NAME} as build arg? Will the manifest substitute the value as correct environment or is the value substituted only at build time?

Yes, you should be able to do that. Copilot will substitute ${COPILOT_ENVIRONMENT_NAME} with the correct environment name before build time - i.e. by the time when it starts building the images, it already knows that wherever ${COPILOT_ENVIRONMENT_NAME} was should be that environment name.

Thanks a lot!

[ssyberg]

👍🏼 would love build time secrets support!