Add a `brew` command to easily verify homebrew-core packages

[josephsweeney]

Verification

• [X] This issue's title and/or description do not reference a single formula e.g. brew install wget. If they do, open an issue at https://github.com/Homebrew/homebrew-core/issues/new/choose instead.

Provide a detailed description of the proposed feature

In a recent PR we started generating build provenance for packages using GitHub's new generate-build-provenance action.

I am proposing and seeking feedback on adding a new brew verify command that would verify installed bottles (either all bottles or individual bottles). The command would use the gh attestation verify extension to the GitHub CLI to verify the installed bottles were signed.

I am currently working on a standalone script to do this but I wanted to get feedback from the maintainers about whether they agree this should be incorporated as a brew command. If there is agreement, please feel free to assign me and I will work on a PR. If not, let me know if there are any changes or further information that would be helpful.

Thank you.

What is the motivation for the feature?

The motivation is to more easily allow users of brew to verify that bottles they install from homebrew-core were indeed built by Homebrew's CI, which is what users already expect. This just allows them to easily verify it without any extra tools.

How will the feature be relevant to at least 90% of Homebrew users?

Any user would be able to use this command to make better security decisions about what bottles to use and trust.

What alternatives to the feature have been considered?

The main alternative to this feature is a standalone command line program that will verify Homebrew bottles. It would do exactly the same thing, but users would have to explicitly find and download the program to verify bottles. This makes it much less likely to be used as it wouldn't be built into brew.

[carlocab]

As a POC, a separate command might be useful. Though: does it make sense to verify the attestation automatically with a brew install?

If doing it on every brew install is too heavy/slow, we could maybe do it randomly for only some packages (and perhaps all the time in Homebrew/core CI for poured bottles).

[josephsweeney]

As a POC, a separate command might be useful.

I agree, my plan is to write a separate command regardless, just to make sure everything works as expected without the complexity of a full brew subcommand. Just wanted to get the conversation started on this early.

Though: does it make sense to verify the attestation automatically with a brew install?

I don't yet know how heavy an operation of verification is, I will investigate to see if this makes sense. Verifying on brew install makes sense in principle, but I figure that can be the next step after we have a standalone command. Then we can work out all the kinks without causing problems with adding steps to a much more widely used subcommand.

[woodruffw]

I don't yet know how heavy an operation of verification is, I will investigate to see if this makes sense.

On the Sigstore side: verification is relatively lightweight (it's a small handful of signature verifications + a bit of parsing and munging). I'll do some benchmarking with sigstore-python today, but I'd expect it to be ~100ms with the extremely slow implementation there (and significantly faster with the Go client, which I believe is what gh attestation verify is built on).

The only potentially slow part is whether verification is done "online," i.e. whether the inclusion proof in each bundle is also checked against the online transparency log (and whether the root of trust is updated on every invocation). The default should be to do an offline verification so no network roundtrip should be necessary there, but the root of trust update may currently be unconditional. I'll look into that!

Edit: From discussion with @phillmv: I forgot to also factor in the network trip for retrieving the attestation itself (which will happen alongside bottle retrieval).

[p-linnane]

Agreed that a brew verify would be nice along with an --online option for doing the online verification.

[MikeMcQuaid]

As a POC, a separate command might be useful.

Yes but would suggest it should be in a external command/tap initially.

On the Sigstore side: verification is relatively lightweight (it's a small handful of signature verifications + a bit of parsing and munging).

👍🏻 my thinking is based on this the verification would go through these steps:

• separate brew verify command in external tap
• brew verify command in Homebrew/brew
• export HOMEBREW_VERIFY_INSTALLS=1 or something does verification at install time
• export HOMEBREW_NO_VERIFY_INSTALLS=1 or something skips verification at install time.

[woodruffw]

That plan SGTM! @josephsweeney, let's start with an external command in a tap on the ToB GH org.

[Bo98]

👍🏻 my thinking is based on this the verification would go through these steps:

• separate brew verify command in external tap
• brew verify command in Homebrew/brew
• export HOMEBREW_VERIFY_INSTALLS=1 or something does verification at install time
• export HOMEBREW_NO_VERIFY_INSTALLS=1 or something skips verification at install time.

Agreed with this though IMO the second step can be skipped as I don't really see any value in doing that over what the external tap offered. If we're interested in onboarding after the experimental stage of being in an external tap, then we're probably interested in fully integrating it into brew install instead of copy-pasting a file and doing nothing really to it.

[josephsweeney]

Agreed on both @MikeMcQuaid's steps and @Bo98's point on the second step. The plan will be to keep brew verify in an external tap (step 1) for a while as an experimental stage. I will keep everyone updated on progress here as I go.

Thanks to everyone for the input, it helps a lot!

[josephsweeney]

I have the first iteration of a brew verify command working in this repo. Please feel free to add the tap and then try running brew verify on any bottles and let me know if you run into any problems! Feel free to comment here, or leave an issue in that repo, whatever is easier.

Note that you'll need the GitHub CLI installed, and this extension added. I explain this in the help text for the command as well. I may add some functionality to prompt the user if they would like to install it, but I figure I would start more minimally for now.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[woodruffw]

@josephsweeney I've outlined the integration steps for this in #17019 and we have the MVP version, so IMO this is safe to close!