Utilize caching when fetching similar repos

[iliazlobin]

Describe the problem/challenge you have When it comes to using overlapping sync options, it's usually that you would mention the same repo to pull from several times in order to substitute one of the folders with a custom one from the private repo, for instance:

directories:
  - path: components
    contents:
      - path: terraform/tfstate-backend
        git:
          url: [email protected]:cloudpossy/terraform-aws-components.git
          ref: all-new-components
        newRootPath: modules/tfstate-backend
      - path: terraform/account
        git:
          url: [email protected]:cloudpossy/terraform-aws-components.git
          ref: all-new-components
        newRootPath: modules/account
      - path: terraform/account-map
        git:
          url: [email protected]:cloudpossy/terraform-aws-components.git
          ref: all-new-components
        newRootPath: modules/account-map
      - path: terraform/nexus
        git:
          url: [email protected]:privateorg/terraform-aws-components.git
          ref: atmos-novel
        newRootPath: privateorg/modules/nexus

Here where it gets too long to wait when vendir sync is used because it basically tries to react on that planly by downloading [email protected]:cloudpossy/terraform-aws-components.git every time it's mentioned in the spec without attempting to re-use the cached sample.

I'd guess, that it's something of utmost importance to implement this simple functionality and make the tool much more efficient.

---

Vote on this request

This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.

👍 "I would like to see this addressed as soon as possible" 👎 "There are other more important things to focus on right now"

We are also happy to receive and review Pull Requests if you want to help working on this issue.

[cppforlife]

@iliazlobin i think it makes sense to add some kind of way to optimize download of same items, however, there isn't a simple way i can see it could be done (the failure mode here is that its very easy to screw cached state). if you want to take a shot at putting some impl ideas down, we could take a look at it and see how gnarly it gets.

[ewrenn8]

I am going to add the carvel-accepted label here since it is something that makes sense to do.