bashbrew icon indicating copy to clipboard operation
bashbrew copied to clipboard

Add "Builder: oci-import" support

Open tianon opened this issue 2 years ago • 1 comments

Closes https://github.com/docker-library/bashbrew/issues/51

In the case of base images (debian, alpine, ubuntu, etc), using a Dockerfile as our method of ingestion doesn't really buy us very much. It made sense at the time it was implemented ("all Dockerfile, all the time"), but at this point they're all some variation on FROM scratch \n ADD foo.tar.xz / \n CMD ["/bin/some-shell"], and cannot reasonably be "rebuilt" when their base image changes (which is one of the key functions of the official images) since they are the base images in question.

Functionally, consuming a tarball in this way isn't that much different from consuming a raw tarball that's part of, say, an OCI image layout (https://github.com/opencontainers/image-spec/blob/v1.0.2/image-layout.md) -- it's some tarball plus some metadata about what to do with it.

For less trivial images, there's a significant difference (and I'm not proposing to use this for anything beyond simple one-layer base images), but for a single layer this would be basically identical.

As a more specific use case, the Debian rootfs.tar.xz files are currently 100% reproducible. Unfortunately, some of that gets lost when it gets imported into Docker, and thus it takes some additional effort to get from the Docker-generated rootfs back to the original debuerreotype-generated file.

This adds the ability to consume an OCI image directly, to go even further and have a 100% fully reproducible image digest as well, which makes it easier to trace a given published image back to the reproducible source generated by the upstream tooling (especially if a given image is also pushed by the maintainer elsewhere).

Here's an example oci-debian file I was using for testing this:

Maintainers: Foo (@bar)
GitRepo: https://github.com/tianon/docker-debian-artifacts.git
GitFetch: refs/heads/oci-arm32v5
Architectures: arm32v5
GitCommit: d6ac440e7760b6b16e3d3da6f2b56736b9c10065
Builder: oci-import
File: index.json

Tags: bullseye, bullseye-20221114, 11.5, 11, latest
Directory: bullseye/oci

Tags: bullseye-slim, bullseye-20221114-slim, 11.5-slim, 11-slim
Directory: bullseye/slim/oci

tianon avatar Dec 03 '22 01:12 tianon

A file the shows off how I discovered that symlinks weren't working:

Maintainers: Foo (@bar)
GitRepo: https://github.com/tianon/docker-debian-artifacts.git
GitFetch: refs/heads/oci-arm32v5
Architectures: arm32v5
GitCommit: f9093e68545e964f67bb3b516017d8ec2033ec71
Builder: oci-import
File: index.json

Tags: bullseye, bullseye-20221114, 11.5, 11, latest
Directory: bullseye/oci

(everything under https://github.com/tianon/docker-debian-artifacts/tree/f9093e68545e964f67bb3b516017d8ec2033ec71/bullseye/oci/blobs/sha256 is symlinks to more usefully-named files so that browsing the directory via GitHub is more user friendly)

tianon avatar Dec 06 '22 22:12 tianon

Codecov Report

Merging #61 (449eb48) into master (18db6c5) will decrease coverage by 7.66%. The diff coverage is 23.15%.

@@            Coverage Diff             @@
##           master      #61      +/-   ##
==========================================
- Coverage   80.77%   73.10%   -7.67%     
==========================================
  Files           6        7       +1     
  Lines         619      714      +95     
==========================================
+ Hits          500      522      +22     
- Misses         94      162      +68     
- Partials       25       30       +5     
Impacted Files Coverage Δ
pkg/gitfs/fs.go 23.15% <23.15%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

codecov-commenter avatar Dec 10 '22 01:12 codecov-commenter

(last push was just a rebase)

tianon avatar Dec 12 '22 23:12 tianon