buildkit
buildkit copied to clipboard
moby
Proposal: Specify standard export and caching APIs
Right now buildkit can export a build in a number of ways, to moby, containerd, etc. I propose to have buildkit define a standard API for handling exports and caching. This could allow a client to specify an export to some external system that buildkit knows nothing about other than that it conforms to buildkit's API. To support new systems for image storage or caching backends no new code needs to be added to buildkit itself and admins/users can implement adapters for their own infrastructure.
This could also allow moby to implement the API such that even a containerized buildkit instance (such as buildx creates) can export images to moby without having to go through the tar/load API. Blobs can be stored directly in moby (I mean, probably in containerd but directed there by moby).
How do you think this API would look like? Is it a blob-only API? Is it like a frontend component that is also loadable today?
How do you think this API would look like? Is it a blob-only API?
I'm not sure without digging into the details of what buildkit needs here. I just wanted to see if people were amenable to this idea.
Is it like a frontend component that is also loadable today?
That is an interesting idea.
Something on the cli like --output=type=docker-image://<ref>?
Yeah I brought up something similar here in a discussion about making exports more modular: https://github.com/moby/buildkit/issues/3037#issuecomment-1235034356
I like this general idea a lot. Between all the code in cache for supporting stargz, nydus, etc. and all the different remotecache implementations, creating something more modular/pluggable could be extremely beneficial to the codebase besides just being more flexible for users.
Thinking about this more, ideally any current exporter should be able to be implemented behind the new "standard" interface. So support for tags would be needed but not required. Probably some other things as well. So not just a blob interface.
This API becomes very useful with the recent changes to the docker-hub pricing.
Since Friday I have been jumping here and there around, trying to find out how to push the images to S3 reliably from our CI hosts. And a possible point-of-deny as a host with a registry:2 container looks redundant there.
