Provide a way to easily update nested bundles

[DennisDenuto]

Describe the problem/challenge you have

an imgpkg bundle with referenced nested bundles can be generalized as a directed acyclic graph. Updating any node in this graph, requires also updating its parent node. The larger the graph the more 'steps' are needed to perform an 'update'

discovering and updating the graph requires multiple steps and is not trivial.

Practically speaking, In order to keep bundles up-to-date custom automation needs to be built by users of imgpkg.

example:

a 'simple' graph might look like this.

rootBundle -> child Bundle A -> child Bundle B

the current workflow to update child Bundle B is:

1. Update child bundle B 1.1) push this bundle and keep reference of the sha
2. Update child bundle A using the sha from 1.1 2.1) push this bundle and keep reference of the sha
3. update rootBundle using the sha from 2.1 3.1) push this bundle

Describe the solution you'd like

A proposed workflow:

1. Update child bundle B 1.1) push this bundle with a tagged semver version (or some other way of declaring a version for a bundle. Perhaps use the bundle.yml meta field to specify a version?)
2. Run an imgpkg command to update the nested bundles.

some strawperson ideas (feel free to burn 'em down):

i.e. imgpkg update -b rootBundle --version major,minor would update rootBundle's images.yml with nested bundles that have a major,minor version bump (based on the tag, or however a version is associated to a bundle)

i.e. imgpkg update -b rootBundle --version latest would update rootBundle's images.yml with nested bundles to the latest version bump (based on the tag, or however a version is associated to a bundle)

Anything else you would like to add: https://kubernetes.slack.com/archives/CH8KCCKA5/p1625001740185600

Open questions / concerns

• How does this work with the version controlled source code of the bundles? -- Perhaps a github action or CI/CD tooling with documentation on carvel.dev is a good alternative approach to this problem?
• does imgpkg update bump the version of the intermediate bundles it updates?
• showing a 'preview' of what it intends to update before pushing the bundles to a registry would be important. To avoid unexpected updates to occur. -- perhaps a --dry-run flag?
• careful consideration will need to be taken around when bundles are 'relocated'. (imgpkg copy) -- how will tags be kept in-sync across registries? (this wouldn't be an issue if we don't capture the version in the tag. i.e the bundle.yml instead)
• relying on tags comes with additional complexity and security implications
• how will this work with tarballs?

---

Vote on this request

This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.

👍 "I would like to see this addressed as soon as possible" 👎 "There are other more important things to focus on right now"

We are also happy to receive and review Pull Requests if you want to help working on this issue.

[jorgemoralespou]

Added my comments that I did on slack:

if you’re doing proper versioning, if you update a version in a bundle, the bundle version should change, and the version of the ones containing that bundle should change again. So, at the end of the day, your problem boils down to how you release and manage your dependencies, and I think it’s not really carvel’s tools problem, but something need to be provided in a pipeline, so that tests and the whole CI/CD can validate the changes made still work. kbld/imgpkg should not take this responsibility. In this scenario, how do you know how many root bundles do contain the bundle you updated?

[DennisDenuto]

The CI/CD workflow you described makes sense to me, and I have the assumption that is the case for most of our users too.

how you release and manage your dependencies

definitely. Although there does seem (to me) to be a gap in the plumbing logic required to make that easily work in a pipeline. That gap places the onus on impkg's users to fill via custom scripts / tooling.

In my head this is analogous to go's module feature. It takes a rather opinionated view on what a valid version is, how to declare a name and version for a module, and also provides lower level 'plumbing' tooling to view the dependency graph and also update dependencies with newer versions too. These lower level go tooling are often used in CI/CD.

and I think it’s not really carvel’s tools problem

I think at the very least, documentation can be provided to outline a workflow for updating nested bundles. It might tease out providing an opinion on what a bundle version is and how it should be declared (is it using tags? a KV in bundle.yml? imgpkg.mod??)

At best, imgpkg provides a similar experience to what go mod provides, meaning it makes it 'easy' for developers, or CI/CD tasks to keep nested bundles up-to-date.

how do you know how many root bundles do contain the bundle you updated?

This is probably rhetorical... but you can't! You would only be able to update starting from a node, and traverse down the graph from there.

[DennisDenuto]

Added the discussion label to open up this issue to also discuss releasing and managing imgpkg's dependencies. Which ties into questions around how to version bundles, enabling workflows around keeping bundles / nested bundles up-to-date.

[joaopapereira]

Since more than 4 months have passed since the last comment in this discussion do we believe this is an issue that we should look into?