Option to append to a ImagesLock when --imgpkg-lock-output is used

[voor]

Describe the problem/challenge you have We have different environments that use different configuration and this shares common manifests, in particular things like Helm charts. We generate our ImagesLock file at the same time we output samples that are committed into the repository for readability purposes (sometimes it can be hard to decipher extremely abstracted ytt templates). This loops through various configurations, for example turning on or off a prometheus metrics exporter or a cluster autoscaler. Since these manifests are all part of the same bundle it'd be nice if they had the same pattern of ./imgpkg/images.yml being referenced in the App CR. However, if the "last" sample isn't using a container then that container is excluded from the ImagesLock file since kbld will not append to an existing one.

Try it yourself:

helm template bitnami/postgresql --set metrics.enabled=true --set postgresqlDatabase=bob | kbld -f - --imgpkg-lock-output images.yml >/dev/null
cat images.yml
helm template bitnami/postgresql --set postgresqlDatabase=bob | kbld -f - --imgpkg-lock-output images.yml >/dev/null
cat images.yml

First output will look something like this:

---
apiVersion: imgpkg.carvel.dev/v1alpha1
images:
- annotations:
    kbld.carvel.dev/id: docker.io/bitnami/postgres-exporter:0.10.0-debian-10-r40
  image: index.docker.io/bitnami/postgres-exporter@sha256:896cb4c4f86d93ceba1f876c4c2ed6206f7d9de911dc29cce291fa2133e966f3
- annotations:
    kbld.carvel.dev/id: docker.io/bitnami/postgresql:11.13.0-debian-10-r12
  image: index.docker.io/bitnami/postgresql@sha256:eab9cdbb7417a51ef17009ed8b90b4f621cb177b54c51d62fedfd571e0398f15
kind: ImagesLock

Second one will be smaller, missing the metrics exporter (which was removed)

---
apiVersion: imgpkg.carvel.dev/v1alpha1
images:
- annotations:
    kbld.carvel.dev/id: docker.io/bitnami/postgresql:11.13.0-debian-10-r12
  image: index.docker.io/bitnami/postgresql@sha256:eab9cdbb7417a51ef17009ed8b90b4f621cb177b54c51d62fedfd571e0398f15
kind: ImagesLock

Describe the solution you'd like Add something like --append-imgpkg-lock-output that would understand there is most likely already a ImagesLock at the provided location to --imgpkg-lock-output and instead of rewriting the file it would just look to see if the provided reference already existed, do a no-op, or update it, or add it to the end.

Anything else you would like to add: If kbld would process this better as just appending full yml files, so you end up with something like:

---
apiVersion: imgpkg.carvel.dev/v1alpha1
---
apiVersion: imgpkg.carvel.dev/v1alpha1
---
apiVersion: imgpkg.carvel.dev/v1alpha1

that's fine too

---

Vote on this request

This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.

👍 "I would like to see this addressed as soon as possible" 👎 "There are other more important things to focus on right now"

We are also happy to receive and review Pull Requests if you want to help working on this issue.

[cppforlife]

@voor could you combine multiple invocations into one?

kbld -f <(helm template bitnami/postgresql --set metrics.enabled=true --set postgresqlDatabase=bob) \
  -f <(helm template bitnami/postgresql --set postgresqlDatabase=bob) \
  --imgpkg-lock-output images.yml >/dev/null

[voor]

Not effectively, since they're also being output to a sample folder and/or part of loops that run at other times.

For the non-helm chart related things, what I've been doing is putting the image as something like:

    spec:
      initContainers:
      - image: change-me-later

And using kbld config:

apiVersion: kbld.k14s.io/v1alpha1
kind: Config
minimumRequiredVersion: 0.24.0
overrides:
- image: change-me-later
  newImage: registry.example.com/project/now-i-am-a-container:latest