flagsmith icon indicating copy to clipboard operation
flagsmith copied to clipboard

Published 20 hours ago • verified publisher icon Flagsmith

Minimise Docker builds in CI

Open khvn26 opened this issue 1 year ago • 6 comments

The following Docker builds are unneccessary:

For E2E:

  • https://github.com/Flagsmith/flagsmith/blob/1e75ae9b573063684c5935fd01ac1945bc869a93/.github/workflows/platform-pull-request.yml#L55-L56
  • https://github.com/Flagsmith/flagsmith/blob/1e75ae9b573063684c5935fd01ac1945bc869a93/.github/workflows/platform-pull-request.yml#L84-L85
  • https://github.com/Flagsmith/flagsmith/blob/1e75ae9b573063684c5935fd01ac1945bc869a93/.github/workflows/platform-pull-request.yml#L113-L114
  • https://github.com/Flagsmith/flagsmith/blob/1e75ae9b573063684c5935fd01ac1945bc869a93/.github/workflows/platform-pull-request.yml#L142-L143
  • https://github.com/Flagsmith/flagsmith/blob/1e75ae9b573063684c5935fd01ac1945bc869a93/.github/workflows/platform-pull-request.yml#L270-L271
  • https://github.com/Flagsmith/flagsmith/blob/1e75ae9b573063684c5935fd01ac1945bc869a93/.github/workflows/platform-pull-request.yml#L293-L294
  • https://github.com/Flagsmith/flagsmith/blob/1e75ae9b573063684c5935fd01ac1945bc869a93/.github/workflows/platform-pull-request.yml#L317-L318
  • https://github.com/Flagsmith/flagsmith/blob/1e75ae9b573063684c5935fd01ac1945bc869a93/.github/workflows/platform-test-merge-to-main.yml#L32-L33

For Uffizzi:

  • https://github.com/Flagsmith/flagsmith/blob/1e75ae9b573063684c5935fd01ac1945bc869a93/.github/workflows/uffizzi-build.yml#L40-L41

All of the above are building the same couple of images. We should build them once per CI run, push to ephemeral registry, and pass the reference around.

khvn26 avatar Apr 04 '24 09:04 khvn26

I think we should tie into this work, the addition of e.g. Docker Scout into our CI pipeline.

https://github.com/docker/scout-action

matthewelwell avatar Apr 17 '24 12:04 matthewelwell

https://github.com/quay/clair-action looks good too

khvn26 avatar May 21 '24 11:05 khvn26

trivy is pretty good — found vulns both from claire's (Quay) and Docker Scout outputs

khvn26 avatar May 21 '24 16:05 khvn26

Still todo: create a dockerfile for our private-cloud image and build this once as part of CI, test against it, and then deploy to SaaS environments as necessary.

matthewelwell avatar Jun 11 '24 15:06 matthewelwell

Related to private-cloud work:

  • https://github.com/Flagsmith/flagsmith-saml/issues/43
  • https://github.com/Flagsmith/flagsmith-auth-controller/issues/12
  • https://github.com/Flagsmith/flagsmith-rbac/issues/20

khvn26 avatar Jun 13 '24 12:06 khvn26

More context for https://github.com/Flagsmith/flagsmith/issues/3718#issuecomment-2161033501:

Having merged the work related to E2E, and significantly reducing Depot costs, we still have workflows that duplicate each other, specifically the private cloud image build and both of the ECS deployments (staging, production). This makes our shipping process prone to human error and hard to comprehend. Additionally, we have a need to use private cloud build for testing in CI, which potentially worsens duplication even more (see #4140).

This can be alleviated by moving the private cloud image build to the Dockerfile, building it once per commit, and reusing the built image for deployments, testing, and shipping. This will eliminate duplication, significantly reduce cognitive load required to maintain the builds, and give us more flexibility to test the private images in various environments — for example, we'll be able to deploy to private cloud straight from the PR to quickly verify a fix, or recommend a customer to deploy latest main branch without having to perform a release.

khvn26 avatar Jun 14 '24 13:06 khvn26