cmd/uses: add `--skip-recursive-build-dependents` flag

[carlocab]

• [x] Have you followed the guidelines in our Contributing document?
• [x] Have you checked to ensure there aren't other open Pull Requests for the same change?
• [x] Have you added an explanation of what your changes do and why you'd like us to include them?
• [ ] Have you written new tests for your changes? Here's an example.
• [x] Have you successfully run brew style with your changes locally?
• [x] Have you successfully run brew typecheck with your changes locally?
• [x] Have you successfully run brew tests with your changes locally?

---

We use brew uses to determine the list of dependent formulae to test in test-bot. By default, we generate this list with the invocation

brew uses --formula --include-build --include-test --recursive <formula>

This results in our testing formulae that do not need to be tested. As an example, suppose foo is a build dependency of bar, and baz depends on bar. Since we don't rebuild bar when testing baz, then no changes in foo are relevant to baz, and any test result from testing baz as a dependent of foo is not meaningful.

One instance where this is particularly wasteful is in go PRs. envoy and [email protected] both declare a build dependency on bazelisk, which in turn declares a build dependency on go. The two formulae also take about four hours each to build.

Let's cut the CI time we spend testing formulae that don't need to be tested by adding a --skip-recursive-dependents flag to brew uses. Adding this flag will omit all dependents of a build dependent from the output.

For reference, here is the list of formulae that are present without the --skip-recursive-build-depenents flag but are no longer there with it:

arduino-cli bat-extras crun envoy [email protected] kube-ps1 kubectx kubernetes-cli kubie modd newrelic-infra-agent [email protected]

[BrewTestBot]

Review period will end on 2021-11-22 at 13:53:28 UTC.

[carlocab]

The implementation in DependenciesHelpers is a bit hacky. Suggestions appreciated.

[MikeMcQuaid]

One instance where this is particularly wasteful is in go PRs. envoy and [email protected] both declare a build dependency on bazelisk, which in turn declares a build dependency on go. The two formulae also take about four hours each to build.

I'm wondering if we should be running brew uses multiple times and do different things with runtime/build/test dependencies. Thoughts?

[carlocab]

I'm wondering if we should be running brew uses multiple times and do different things with runtime/build/test dependencies.

Possibly, yes. We don't need to build test dependents from source, for example. I think we might still be doing that.

[carlocab]

This isn't quite correct yet.

❯ brew uses --include-build --include-test --recursive --skip-recursive-build-dependents --formula [email protected]
arduino-cli              kops                     kubectx                  kubie                    modd                     tanka
devspace                 krew                     kubeless                 landscaper               newrelic-infra-agent     [email protected]
helmsman                 kube-ps1                 kubernetes-cli           minikube                 riff

devspace depends on kubernetes-cli (runtime) and go (build). kubernetes-cli depends on [email protected] (build). My intention is for us to not see devspace here, since it is a dependent of a build dependent.

[carlocab]

Better:

❯ brew uses --include-build --include-test --recursive --formula --skip-recursive-build-dependents [email protected]
arduino-cli                   kubernetes-cli                modd                          newrelic-infra-agent          [email protected]

❯ comm -23 <(brew uses --include-build --include-test --recursive --formula go) <(brew uses --include-build --include-test --recursive --formula --skip-recursive-build-dependents go)
arduino-cli
bat-extras
cdktf
charm-tools
conjure-up
crun
envoy
[email protected]
git-annex-remote-rclone
juju-wait
kube-ps1
kubectx
kubernetes-cli
kubie
localstack
modd
navi
newrelic-infra-agent
snapcraft
[email protected]
wireguard-tools

[carlocab]

Bonus: this will cut 510 (!) formulae from a [email protected] CI run.

https://gist.github.com/carlocab/5aa6e3f05e4bad513f1cd11fb619e69e

Will check a few of these to make sure this is the behaviour we want.

[BrewTestBot]

Review period ended.

[github-actions[bot]]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[carlocab]

Reopening after discussion at Homebrew/homebrew-test-bot#837. Conflicts need resolving. Also some discussion here about correctness/desirability.

[github-actions[bot]]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.