`podman manifest create` should support `--format`

[jlebon]

trafficstars

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind feature

Description

In OCP-land, we need to deal with some clients that don't yet support OCI manifest lists. Because of this, we push them in v2s2 format. Our tooling needs to be able to know the final digests of the constituent images. We currently do this using podman manifest inspect. However, if we push in v2s2 format, the digests on the remote will unsurprisingly not match.

Right now, we work around this by doing a skopeo inspect --raw of the target repo right after pushing it to get the final digests, but ideally we wouldn't have to do that. The problem is that there's no way to create the manifest in v2s2 format right from the start.

[rhatdan]

@flouthoc PTAL

[github-actions[bot]]

A friendly reminder that this issue had no activity for 30 days.

[jlebon]

Still interested in this.

[github-actions[bot]]

A friendly reminder that this issue had no activity for 30 days.

[jlebon]

Still interested in this.

[flouthoc]

I'll check this thanks.

[github-actions[bot]]

A friendly reminder that this issue had no activity for 30 days.

[jlebon]

Still interested in this.

[varshneydevansh]

Is flouthoc still working on this? If not can I :)

[rhatdan]

Yes go ahead. Flouthoc is not a full time student, so he has limited time to contribute. You can take it over.

[varshneydevansh]

I'll look into this soon

[lslavkov]

Can I ask what is the status of this?

[varshneydevansh]

I will do this as first I have to deliver the X and Y axis changes for this https://gerrit.libreoffice.org/c/core/+/167068

[varshneydevansh]

I will do this, I already forked it but got stuck with my friends' thesis project.

[jankaluza]

I will take this one.

[mtrmac]

In OCP-land, we need to deal with some clients that don't yet support OCI manifest lists. Because of this, we push them in v2s2 format. Our tooling needs to be able to know the final digests of the constituent images. We currently do this using podman manifest inspect. However, if we push in v2s2 format, the digests on the remote will unsurprisingly not match.

This is conceptually not how the tools’ workflows work. podman push typically changes digests. For locally-pulled images, it almost always does so. On every format conversion, it does so. Conversions can be triggered automatically by a registry refusing to accept a manifest in the first attempted format.

Nothing in Podman “stages” the final form of images in a way that can be “inspected” in advance. (podman … inspect also, often, does not even output data in a format equivalent to on-registry representations. It exists for users, not for registries or data storage.)

I realize that the inspecting works in that one particular setup (probably with a known destination registry that is known to accept OCI, and no other modifications of the images); I don’t think this is a model that podman manifest can be built around.

---

More specifically, if I understand the situation correctly, the per-platform components originally exist in OCI, and the goal is to convert them to v2s2 and push elsewhere. The conversion happens during the push. This is fundamental to the model of podman manifest being a metadata-only operation where data is streamed directly from the source registry to the destination (or, ideally, only linked within the same registry). If podman manifest inspect were to be able to report the final v2s2 digests, podman manifest would have to:

• Pull the OCI per-platform images to local storage (unnecessary otherwise)
• Convert the per-platform images into a v2s2 form in $some (currently nonexistent) staging location, including possibly re-compressing them if they are originally Zstd (unnecessary otherwise, and pretty costly)

Besides the infrastructure outright not existing, it’s unclear when that cost should be paid:

• it would be very unexpected for podman manifest inspect to stall for a few minutes
• podman manifest add does not know whether podman manifest inspect is coming = whether it should do the costly work.

---

“Push, then see what is on the registry” is conceptually the right model when conversions / recompressions are involved. Both podman push and podman manifest push have a --digestfile option that allows identifying exactly what was pushed. If you need not just the top-level digest, but also digests of the per-platform components, --digestfile provides an immutable reference that allows looking up the per-platform components.

[jlebon]

@mtrmac Thanks, that helps. We'll stick with --digestfile.