compose icon indicating copy to clipboard operation
compose copied to clipboard

Build vendor artifact

Open catap opened this issue 1 year ago • 11 comments

Closes: https://github.com/docker/compose/issues/11760

catap avatar Apr 24 '24 10:04 catap

@ndeloof here it is

catap avatar Apr 24 '24 10:04 catap

LGTM but should be declared in the merge workflow so this run for commits on main branch and tags

ci workflow is triggered for by push into main brnach and tags as well. Also, this workflow makes all artifacts and it seems for me logical that it should be used to produce this one.

Have I missed something?

catap avatar Apr 24 '24 10:04 catap

ci workflow is triggered by push to main but also by pull_requests, and this doesn't make sense to build this src archive for PRs merge workflow is used for stuff to only happen on main branch and releases, like building bin image

ndeloof avatar Apr 24 '24 10:04 ndeloof

We should probably check if there's additional verification needed for these packages. The standard source tarballs on GitHub releases are produced from git, and therefore should always match the source code. This PR constructs a custom tar; while it's produced through GitHub actions, it's effectively still a "manual" upload, so we should check if there's ways to verify provenance and integrity.

Recent events with the xz backdoor showed that manually / custom uploaded sources were at the root of that backdoor (the xz project instructed users to not use the default tarballs produced by GitHub, but the ones that were manually uploaded).

cc @rachel-taylor-docker @gabriellavengeo

thaJeztah avatar Apr 24 '24 11:04 thaJeztah

@thaJeztah this is one of root cause why I'm doing this.

Right now OpenBSD port for docker-compose depends on .tar.gz which I build and host on my machine. Everyone should trust that I haven't been compromised and doesn't inject something very bad into this .tar.gz and has we seen such injection can be quite complicated.

So, the right way to do it is download from the same source where it sources is produced becuase this should be trusted source by defenition.

Regarding the way: as soon as I know go mod vendor respects checksums inside go.mod that should prevent from packing bad things. Or, let me say it that way: it should pack the same things which user should get if he run it locally.

catap avatar Apr 24 '24 11:04 catap

ci workflow is triggered by push to main but also by pull_requests, and this doesn't make sense to build this src archive for PRs merge workflow is used for stuff to only happen on main branch and releases, like building bin image

To keep things simpler I've added if inside this step.

Otherwise I need to move and duplicate code inside merge workflow which seems wrong from my point of view.

catap avatar Apr 24 '24 12:04 catap

Right now OpenBSD port for docker-compose depends on .tar.gz which I build and host on my machine. Everyone should trust that I haven't been compromised and doesn't inject something very bad into this .tar.gz and has we seen such injection can be quite complicated.

Yes, anyone producing binaries / artifacts for others to consume must have provenance. I think that goes beyond just these tars, and there would be many other ways for that build-environment / infra to be comprimised.

Regarding the way: as soon as I know go mod vendor respects checksums inside go.mod that should prevent from packing bad things. Or, let me say it that way: it should pack the same things which user should get if he run it locally.

Correct; it won't protect against a compromised tarball that has go.mod and go.sum tampered with though; so I'm looking if there's any way to have provenance for the uploaded tarball to verify if it's produced by the GitHub action, and not replaced in other ways.

The default tarballs produced by GitHub always match the git-commit of the release, but for other artifacts that's not the case, so some validation may be necessary for that.

thaJeztah avatar Apr 24 '24 14:04 thaJeztah

Regarding the way: as soon as I know go mod vendor respects checksums inside go.mod that should prevent from packing bad things. Or, let me say it that way: it should pack the same things which user should get if he run it locally.

Correct; it won't protect against a compromised tarball that has go.mod and go.sum tampered with though; so I'm looking if there's any way to have provenance for the uploaded tarball to verify if it's produced by the GitHub action, and not replaced in other ways.

The default tarballs produced by GitHub always match the git-commit of the release, but for other artifacts that's not the case, so some validation may be necessary for that.

I haven't see any differences with producing .tar.gz by this job with producing binary artifacts by the same github actions.

Second point: GitHub and other git hosting sometimes changes produced tar.gz from tag and they officialy confirm it, see: https://github.blog/2023-02-21-update-on-the-future-stability-of-source-code-archives-and-hashes/ and to avoid such mess they suggested to attach artifacts to releases.

catap avatar Apr 24 '24 15:04 catap

just a ping

catap avatar Apr 30 '24 14:04 catap

Can you look into https://docs.github.com/en/actions/security-guides/using-artifact-attestations-to-establish-provenance-for-builds#generating-build-provenance-for-binaries so this additional artifact gets provenance attestation and you can safely rely on this for an offline build ?

ndeloof avatar May 06 '24 12:05 ndeloof

On Mon, 06 May 2024 13:54:21 +0100, Nicolas De loof @.***> wrote:

Can you look into https://docs.github.com/en/actions/security-guides/using-artifact-attestations-to-establish-provenance-for-builds#generating-build-provenance-for-binaries so this additional artifact gets provenance attestation and you can safely rely on this for an offline build ?

Thanks for the link.

I've read it and I doesn't see how it can help me.

OpenBSD ports, for example, ships its own checksum of artifacts which should be used to build port.

It is checked by build servers which downloads artifacts from github.com or other plases and confirms that checksum holds.

-- wbr, Kirill

catap avatar May 06 '24 18:05 catap