Builds with multiple --cache-from images

[richafrank]

We're using this to --cache-from intermediate and final stages of a multi-stage docker build, instead of busting the cache every time on building the intermediate target.

Same as https://github.com/buildkite-plugins/docker-compose-buildkite-plugin/pull/271 , but re-opening to try to trigger the buildkite webhook for the PR.

[nsuma8989]

@richafrank Added comments above which are causing the build to fail. Please can you look into those when possible.

[richafrank]

@nsuma8989 I just pushed a linter fix. Is there a way for me to trigger a buildkite build when I have changes to test?

[nsuma8989]

@richafrank Option to trigger new builds for open PR and third-party repos is disabled so it is not triggering build. I can trigger a build for your new commits like a did for the latest commit.

linter fix did not fully address it based on the latest build.

[richafrank]

Ok, thanks @nsuma8989 , I'll ping you about builds as I need.

Looks like the linter issue is that this plugin no longer supports building and pushing in the same step of the pipeline. Is that intentional? We had been using that ability to great effect, relying on the fact that the pushing agent was guaranteed to have the image ready. Is the expectation now that one agent will build the image and another might end up pushing it?

[nsuma8989]

Ok, thanks @nsuma8989 , I'll ping you about builds as I need.

Looks like the linter issue is that this plugin no longer supports building and pushing in the same step of the pipeline. Is that intentional? We had been using that ability to great effect, relying on the fact that the pushing agent was guaranteed to have the image ready. Is the expectation now that one agent will build the image and another might end up pushing it?

@richafrank Based on the linter error, it is actually complaining about missing required parameter "run"

  [2022-04-25T00:54:28Z]         - keyword: required
  | [2022-04-25T00:54:28Z]           dataPath: ""
  | [2022-04-25T00:54:28Z]           schemaPath: "#/oneOf/0/required"
  | [2022-04-25T00:54:28Z]           params:
  | [2022-04-25T00:54:28Z]             missingProperty: run
  | [2022-04-25T00:54:28Z]           message: should have required property 'run'

[richafrank]

Thanks @nsuma8989 . Please correct me if I'm misunderstanding. I see two errors reported by the linter. The first is as you said, and the second is:

[2022-04-25T00:54:28Z]         - keyword: oneOf
[2022-04-25T00:54:28Z]           dataPath: ""
[2022-04-25T00:54:28Z]           schemaPath: "#/oneOf"
[2022-04-25T00:54:28Z]           params:
[2022-04-25T00:54:28Z]             passingSchemas:
[2022-04-25T00:54:28Z]               - 1
[2022-04-25T00:54:28Z]               - 2
[2022-04-25T00:54:28Z]           message: should match exactly one schema in oneOf

If I add a dummy run parameter to solve the first issue:

diff --git a/README.md b/README.md
index ceed063..1e048b3 100644
--- a/README.md
+++ b/README.md
@@ -328,23 +328,25 @@ steps:
           cache-from:
             - myservice_intermediate:index.docker.io/myorg/myrepo/myservice_intermediate:${BUILDKITE_BRANCH}
             - myservice_intermediate:index.docker.io/myorg/myrepo/myservice_intermediate:latest
+          run: test
           push:
             - myservice_intermediate:index.docker.io/myorg/myrepo/myservice_intermediate:${BUILDKITE_BRANCH}
             - 

the linter still reports the second error, which I believe is saying that only one of run, build, push is allowed. This looks to be schema validation that's newer and more restrictive than when I opened the original PR. Is it required?

[nsuma8989]

Looks like the linter issue is that this plugin no longer supports building and pushing in the same step of the pipeline. Is that intentional? We had been using that ability to great effect, relying on the fact that the pushing agent was guaranteed to have the image ready

@richafrank Thank you for fixing the first issue in the linter. For second issue, validation schema was always using "OneOf" as you can see from this initial PR https://github.com/buildkite-plugins/docker-compose-buildkite-plugin/pull/119. So schema validation did not change recently. Please can you confirm that you are able to use build and push in single step using this plugin ?

[toote]

Closing this one as it is the same (and contained in) #328