build-push-action icon indicating copy to clipboard operation
build-push-action copied to clipboard

Multi platform build with caching?

Open SebastianStehle opened this issue 2 years ago • 6 comments

Hi,

this is more a discussion. Have you been able to get multi-platform build running? When I just enable it I get the following error:

https://github.com/Squidex/squidex/actions/runs/3581565934

When I turn of load: true, then I get another exception:

WARNING: No output specified with docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load

I found a lot of bug reports in the docker repo regarding that, e.g. https://github.com/docker/buildx/issues/1220

SebastianStehle avatar Nov 30 '22 11:11 SebastianStehle

ERROR: docker exporter does not currently support exporting manifest lists

You're trying to load a multi-platform image to the Docker store which is not supported atm.

When I turn of load: true, then I get another exception:

If you don't load then the image result has nowhere to go, that's why you have this warning. You can either push the image to a registry like you've done: https://github.com/docker/build-push-action/issues/733#issuecomment-1332263462


Looking at the workflow, the build job looks to just warm-up cache so you don't need to rebuild the image in dependent test and publish jobs.

If that's the case you should just build the image and export cache using the gha exporter and load back the cache in the other jobs. So you can remove:

  • https://github.com/Squidex/squidex/blob/9d85b5fb55743b7c6779b3817f97a3e607044ef4/.github/workflows/release.yml#L25-L31
  • https://github.com/Squidex/squidex/blob/9d85b5fb55743b7c6779b3817f97a3e607044ef4/.github/workflows/release.yml#L43-L56

For the test job you need a step to build the image and import cache from gha. As you're testing the image in this job you need to load to the store. See https://docs.docker.com/build/ci/github-actions/examples/#test-your-image-before-pushing-it

crazy-max avatar Nov 30 '22 15:11 crazy-max

Thank you for your answer. I have picked another project, which in general just builds faster. I use the registry as cache and then pull it from the registry:

https://github.com/notifo-io/notifo/blob/multiplatform/.github/workflows/dev.yml#L36

This seems to work so far.

SebastianStehle avatar Nov 30 '22 15:11 SebastianStehle

I use the registry as cache and then pull it from the registry:

It will not work on pull request as you need to be authenticated against the registry to push the cache. But gha will work.

crazy-max avatar Nov 30 '22 15:11 crazy-max

Good point. I could just get rid of the build steps. but it is easier to understand the issue when the build fails.

SebastianStehle avatar Nov 30 '22 15:11 SebastianStehle

I hope it is okay to ask that here. Is it possible to run a stage shared between platforms? There is no value to build my React frontend twice.

SebastianStehle avatar Nov 30 '22 15:11 SebastianStehle

I tried your approach with another PR and now I am confused.

  1. When load == false I cannot load the image after I have built it: https://github.com/notifo-io/notifo/actions/runs/3585260426/jobs/6033249244#step:12:57
  2. When load == true the build would fail with multiplatform enabled.
  3. When I pull later, docker would probably only pull one platform, and not for all platforms. So I need to rebuild again anyway.

So my only option seems to be...

  1. Build: Push to a temporary tag
  2. Test: Pull from temporary tag.
  3. Publish: Rebuild again to be able to set new tags for all platforms.

SebastianStehle avatar Nov 30 '22 17:11 SebastianStehle

I hope it is okay to ask that here. Is it possible to run a stage shared between platforms? There is no value to build my React frontend twice.

Please open a new issue for this thx.

crazy-max avatar Mar 08 '24 14:03 crazy-max