mina icon indicating copy to clipboard operation
mina copied to clipboard

Build docker images using cloud build

Open balsoft opened this issue 2 years ago • 9 comments

Use cloud build for releasing docker images. Generate cloudbuild.yaml with dhall, then call google builds submit Fixes #10693

balsoft avatar May 12 '22 08:05 balsoft

Need to add dhall-json to the toolchain for CI to work

balsoft avatar May 17 '22 11:05 balsoft

This PR should be converted to a 'draft' because the CI is entirely red and because the descriptions still says "WIP" on its first line.

robinbb avatar May 19 '22 20:05 robinbb

Do you have a sense of how to re-use the dhall logic for a simple "docker build" command locally (with some flag to release-docker.sh)?

That's a great suggestion, I'll try to implement it.

balsoft avatar May 20 '22 10:05 balsoft

@lk86 Done, please re-review if you have time!

balsoft avatar May 20 '22 10:05 balsoft

PR adding Dhall to toolchain: #11044

balsoft avatar May 23 '22 14:05 balsoft

As I understand it, this PR offloads docker image creation to google cloud build (which spawns a new VM and uses kaniko to build the dockerfile there).

  • What happens if the build is interrupted or preempted? Should a timeout be set?
  • Where do logs go?
  • What does this cost?

yorickvP avatar May 24 '22 07:05 yorickvP

@balsoft Please update this PR with descriptions of why the CI is red and why you are blocked on this.

robinbb avatar Jun 15 '22 20:06 robinbb

It's red because of some problem the nature of which escapes me. I'm currently investigating it.

balsoft avatar Jun 15 '22 20:06 balsoft

The image BuildKite currently runs on has a pretty old version of the google cloud CLI. I'd first have to figure out how to upgrade that before I can proceed testing out this approach. The reason I'd need to update it, is because the currently installed versions doesn't support the --worker-pool flag.

bryanhonof avatar Aug 31 '22 11:08 bryanhonof

What is the status on this @bryanhonof @robinbb

lk86 avatar Oct 25 '22 06:10 lk86

It looks like @bryanhonof worked on this for a while and then stopped. @bryanhonof can answer, and maybe @kantp was involved in de-prioritizing? This is a PR, so if it is not ready for review (and it is not, because the CI is red), then it should be marked draft.

robinbb avatar Oct 25 '22 20:10 robinbb

@robinbb @kantp this work has a lot of value to O(1) and last I talked to Bryan and tweag they were making great progress, this is the first I hear that is been de-prioritized. Can we discuss the motivation for that and what direction you are intending to go instead?

lk86 avatar Oct 25 '22 21:10 lk86

I deduced the de-prioritization based on the lack of progress.

robinbb avatar Oct 25 '22 21:10 robinbb

I am interested in the same answers as @lk86 .

robinbb avatar Oct 25 '22 21:10 robinbb

!ci-build-me

bryanhonof avatar Dec 02 '22 10:12 bryanhonof

!ci-build-me

lk86 avatar Dec 10 '22 07:12 lk86

!ci-build-me

lk86 avatar Dec 10 '22 07:12 lk86

!ci-build-me

lk86 avatar Dec 10 '22 08:12 lk86

!ci-build-me

lk86 avatar Dec 10 '22 09:12 lk86

!ci-build-me

lk86 avatar Dec 10 '22 09:12 lk86

!ci-build-me

lk86 avatar Dec 10 '22 09:12 lk86

!ci-build-me

lk86 avatar Dec 10 '22 09:12 lk86

!ci-build-me

lk86 avatar Dec 10 '22 09:12 lk86

!ci-build-me

lk86 avatar Dec 10 '22 09:12 lk86

!ci-build-me

lk86 avatar Dec 10 '22 10:12 lk86

!ci-build-me

lk86 avatar Dec 10 '22 10:12 lk86

!ci-build-me

lk86 avatar Dec 10 '22 10:12 lk86

!ci-build-me

lk86 avatar Dec 10 '22 11:12 lk86

!ci-build-me

lk86 avatar Dec 10 '22 11:12 lk86

!ci-build-me

lk86 avatar Dec 10 '22 11:12 lk86