cache_from parameter does not work

[jacob-hd]

When using cache_from and providing a docker image with layers that do not need to be re-built, Concourse will re-build all of the layers every time instead of using cached layers. (It is the same behavior as if you did not use cache_from at all.)

Here is an example pipeline:

---
resource_types:
- name: pull-request
  type: docker-image
  source:
    repository: jtarchie/pr

resources:
- name: pull-request
  type: pull-request
  source:
    repo: quotecenter/app-thunder-storefront
    access_token: ((github-token))
    username: x-oauth-basic
    password: ((github-token))

- name: docker-image
  type: docker-image
  source:
    repository: gcr.io/np-quotecenter/app-thunder-storefront/pull-requests
    username: _json_key
    password: ((gcr-key))

- name: docker-image-master
  type: docker-image
  source:
    repository: gcr.io/np-quotecenter/app-thunder-storefront/master
    username: _json_key
    password: ((gcr-key))
    tag: latest

jobs:
- name: build
  plan:
  - aggregate:
    - get: app-thunder-storefront
      resource: pull-request
      trigger: true
      version: every

    - get: docker-image-master
      params:
        save: true

  - put: docker-image
    params:
      build: app-thunder-storefront
      build_args:
        NPM_TOKEN: ((myget-token))
      tag: app-thunder-storefront/.git/head_sha
      cache_from: [ docker-image-master ]

The docker image docker-image-master contains the exact same docker image that this PR is building - I have verified this several times.

The resulting output shows that it builds the entire image from scratch without using any caching:

[jacob-hd]

Note that if I jury-rig the setup to use the same docker image, but via cache: true and cache_tag: latest instead of the cache_from array, it does use the cached layers:

It is only cache_from that is broken.

Thanks for your time in this matter.

[mattftw]

I'm experiencing this issue as well. Is it possible we're using this feature incorrectly?

[jacob-hd]

@mattftw I'm pretty confident that we are using it correctly.

[adarshaj]

I have just used this with latest concourse, and this features works just fine. one thing that might be a problem is if your concourse version itself is old (because this docker resource is bundled)? other thing that can be done is to introduce a do with get followed by put (in serial steps) instead of aggregate in the first step of build.

[chrishiestand]

Another possibility is if your image is a multi-stage build. If so, you need to cache_from the intermediate images. https://github.com/concourse/docker-image-resource/issues/148#issuecomment-402879655

[jacob-hd]

This is not a multi-stage build, but thanks for the troubleshooting.

[dhinus]

HI @jacob-hd, I added the cache_from param to the Docker resource and it's working for me. Could you please post the first part of the job output? Before step 1/23 it should print something like Loaded image ID: sha256:dd574b216ad79585d75fa359fc742103adfeaddc3176f8524cc6f7e5818d7248.

It if isn't, it looks like it's ignoring the cache_from argument. Can you double check the version of the worker you're using? You can also try to override the bundled Docker resource by pointing to the Git repo directly:

resource_types:
- name: docker-image
  type: docker-image
  source:
    repository: concourse/docker-image-resource

[xtremerui]

@jacob-hd is this still a problem for you with new concourse? May I close this issue? Thx.