Please add opm render support for bundle directories.

[J0zi]

Hi team,

opm render is able to consume bundle image. But we are missing the feature to render bundle directory as an input. So directly processing directory which contains CSV and CRDs.

Our source of truth is git repository and final product is an index image. Also customers can make an advantage of the new would be feature and generate file based config directly from a source code.

Implementation: something like this

opm render <path-to-bundle-dir>  --link-to-bundle=<path-to-bundle-image> --output=yaml >> <some-filename>.yaml

will create file based config. Also path-to-bundle-image will be added to relatedImages, with no check if it exists or if there is some content. Just the value will be inserted like this:

relatedImages:
- image: quay.io/openshift-community-operators/sosreport-operator:v0.0.2
  name: ""

Feel free to find a better name for link-to-bundle parameter.

Many thanks. Community operators maintainers

[joelanford]

Ah yes, I had forgotten that the problem with rendering bundle directories is the lack of a known bundle image.

I'm hesitant to add a flag for that though, for a few reasons:

1. render supports other types of input (sqlite dbs/images, fbc directories/images) and that flag wouldn't make sense in the context of those other input types.
2. render supports multiple inputs as arguments, so when dealing with something like opm render <bundleDir1> <bundleDir2>, that flag would be ambiguous and not very ergonomic.

Would it be acceptable to delegate the task of inserting image and relatedImage values to another command that can take the output of render via stdin? For example:

opm render <path-to-bundle-dir> -o yaml | \
	yq  e '.image="<path-to-bundle-image>" | .relatedImages += [{"image": "<path-to-bundle-image>", "name":""}]' -

[J0zi]

@joelanford It would be perfect for user to enter just bundle path and leave tricks for opm to handle image path string to the correct line. What do you think?

[joelanford]

@J0zi One of the big reasons for the switch to FBC is to take some of the magic out of opm and make it much more predictable than what currently exists with the opm index/registry commands. So IMO, we'd be setting the wrong precedent if we started making the new FBC tooling handle these kind of one-off tricks.

In this case, my specific concern with the suggested approach is that it doesn't mesh will with the design of the render CLI

The original suggestion here seems pretty straighforward and logical:

opm render <path-to-bundle-dir>  --link-to-bundle=<path-to-bundle-image>

But what about these:

opm render <path-to-bundle-image>  --link-to-bundle=<path-to-bundle-image>
opm render <path-to-bundle-dir> <path-to-another-bundle-dir>  --link-to-bundle=<path-to-bundle-image>
opm render <path-to-bundle-image> <path-to-bundle-dir>  --link-to-bundle=<path-to-bundle-image>
opm render <path-to-index-image>  --link-to-bundle=<path-to-bundle-image>
opm render <path-to-index-image> <path-to-bundle-dir>  --link-to-bundle=<path-to-bundle-image>

In each of these cases, it either doesn't make sense to specify the --link-to-bundle flag, or it does make sense, but ambiguity creeps in because it isn't clear which rendered thing to inject the provided bundle image into.

[J0zi]

@joelanford opm render <path-to-bundle-dir> --link-to-bundle=<path-to-bundle-image> could render from directory and add only following stanza:

relatedImages:
- image: quay.io/openshift-community-operators/sosreport-operator:v0.0.2
  name: ""

So it is adding missing puzzle there, path to the bundle. It is valuable for us.

[porridge]

In my hacked version I managed to overcome the problem of conveying the directory name and the image name in a single argument by requiring the directory to be called the same as the image :joy: It's beyond ugly, but perhaps could serve as an inspiration for some cleaner approach?

Perhaps something like opm render <bundle-image-spec>[@<path-to-bundle-dir>] [...]?

[joelanford]

The other issue to overcome (as that hacked commit highlights) is that we'd need a way to distinguish a bundle directory from an FBC directory.

[porridge]

FBC is file-based catalog? We distinguish a bundle image from index image using labels I guess? I don't know the format by heart, but I guess there is some distinguishing feature between the content of a bundle directory and an FBC directory that can be used?

[joelanford]

I know it has been quite a long time, but I re-opened #748, rebased it and added a --image-ref-template flag that can accept a template string with {{.Package}}, {{.Name}}, and {{.Version}}.

This solves the problem of trying to figure out a CLI UX that involves rendering multiple bundles. I think the open question in my mind is whether package, name, and version provide enough granularity to generate useful image references.

@J0zi @porridge, WDYT?

[cdjohnson]

Just stating the obvious: The bundle image references must use tags and can't use digests.

[joelanford]

Correct. Ultimately, the references templated with --image-ref-template would essentially be placeholder strings. A pipeline could try to build and push using that reference, but I imagine that the more likely use case is that a pipeline would build and push using its own methodology, and then have a way to understand how to map the source image references generated by --image-ref-template to the destination image reference that it pushed, such that it could use the destination image ref and do SHA pinning at release artifact build time.

[joelanford]

We have merged support for rendering bundle directories as an alpha feature in #748.

For now, this is subject to change or be removed at any time, but if you are interested in this feature, please try it out and provide feedback, positive or negative!