brew
brew copied to clipboard
Homebrew
formula_auditor: audit for Linux-only dependencies on GCC
- [x] Have you followed the guidelines in our Contributing document?
- [x] Have you checked to ensure there aren't other open Pull Requests for the same change?
- [x] Have you added an explanation of what your changes do and why you'd like us to include them?
- [ ] Have you written new tests for your changes? Here's an example.
- [x] Have you successfully run
brew stylewith your changes locally? - [x] Have you successfully run
brew typecheckwith your changes locally? - [ ] Have you successfully run
brew testswith your changes locally?
As we've seen, allowing Linux-only dependencies on GCC results in its dependency tree growing out of control to the point of being extremely painful to maintain.
Let's stop this situation from getting worse by:
- failing a
--strictaudit when there is a Linux-only GCC dependency. This also prevents new formulae from having such a dependency. - failing any audit whenever a formula that did not previously have a Linux-only GCC dependency picks one up
If a future formula update causes a formula to fail to build on Linux because it needs a newer GCC when it previously did not, then we should not bottle it. We can bottle this hypothetical formula when our bottling distribution includes a new enough version of GCC.
Requesting review from @Homebrew/core, since this might be a pretty hard pivot from bottle coverage being one of our highest, if not the highest, priorities. As I've said elsewhere, let's start pushing maintainability up that list.
We can bottle this hypothetical formula when our bottling distribution includes a new enough version of GCC.
I think this is also useful as a kind of commitment device for making sure our bottling distribution is updated.
Sorry but I don’t agree at all with this as a strict requirement. We have made the necessary changes for Linux-only GCC dependencies not be disruptive to GCC upgrades and I think we should retain the flexibility to allow this if needed. If we use a new a enough image we will rarely need to resort to using GCC on Linux, so the very occasional exception isn’t going to cause an issue.
If we get onto 22.04 as I hope we will, I think we should see how many formulae end up needing Linux-only GCC dependencies in the 2 years we use it. If that number stays very low, then this is a non-issue. But until we have that data, I think we should take such a drastic decision.
We have made the necessary changes for Linux-only GCC dependencies not be disruptive to GCC upgrades and I think we should retain the flexibility to allow this if needed.
@danielnachun Can you elaborate on that? Is it that revisions will no longer be required or that dependency/recursive dependency testing will no longer be required? If the prior: would you have confidence in making CI avoid the latter?
If we get onto 22.04 as I hope we will, I think we should see how many formulae end up needing Linux-only GCC dependencies in the 2 years we use
I'd like a way to stop the bleeding until we get to 22.04 and then we can reconsider this audit.
I'd like a way to stop the bleeding until we get to 22.04 and then we can reconsider this audit.
This is my primary motivation for this PR.
I have more to add here, but a bit busy at the moment. Will get back to this in the next few days.
I agree with @carlocab: we need to stop the creep of dependencies, then once the migration to the new linux build environment is done, we can reconsider. It's safer than the other way around. The previous creep of linux-only gcc dependencies several impacted the dependency tree, and we need to avoid that to continue (even though we have a stop gap measure).
We should absolutely skip dependent testing of Linux-only GCC dependencies - revision bumps should never be needed now. Now that we use the "current" symlink, we can't break bottles on Linux on GCC updates. Better yet, we won't even have to revision bump gfortran dependencies that are shared across Linux and macOS unless the soname for libgfortran is bumped, which doesn't happen with every major release. So GCC updates should be much less painful now on both platforms.
I also don't expect the dependency tree to grow much now - there are only about 70 unbottled formulae on Linux and only a couple of those could need brewed GCC, but they have unrelated issues blocking their bottling. We may pick up a new dependency here or there if a new formula is added but it seems unlikely.
If we just skip dependent testing, the only annoyance that could happen is if an existing formula picks up a brewed GCC dependency and it has C++ dependents that would propagate that dependency to them. But that would affect the update to that formula, not updating GCC.
So I think the best "stop gap" measure here is to basically ignore the Linux-only GCC dependencies when updating the GCC formula, which would restore things back to how they were before. Though I think that is actually a change we should make permanent, as there is really no point in dependent testing anything that relies on libstdc++ - it's deliberately designed not to need this, and the only reason it gave us so much trouble was an oversight in our GCC formula that is now solved.
revision bumps should never be needed now
Sorry to be a pedant but is this "should never" "will never" or "I hope will never"?
I also don't expect the dependency tree to grow much now
The "grow much" is a concern here. @carlocab @fxcoudert and I seem to all want it to not "grow at all".
So I think the best "stop gap" measure here is to basically ignore the Linux-only GCC dependencies when updating the GCC formula, which would restore things back to how they were before.
I'm potentially fine with this until we get a single user-submitted issue about breakage.
As-is, though, it does seem we have more folks in favour of this than opposed. The quicker we get a 22.04 migration out: the quicker this check is going to be less or not needed.
Sorry to be a pedant but is this "should never" "will never" or "I hope will never"?
I am going to say "will never".
I'm potentially fine with this until we get a single user-submitted issue about breakage.
That makes sense. We did get a full test of this in CI in the GCC 12 PR and everything looked great so if we had breakage it would have to be a weird corner case.
Sorry to be a pedant but is this "should never" "will never" or "I hope will never"?
I am going to say "will never".
Cool, thanks 👍🏻
I'm potentially fine with this until we get a single user-submitted issue about breakage.
That makes sense. We did get a full test of this in CI in the GCC 12 PR and everything looked great so if we had breakage it would have to be a weird corner case.
Ok, fine with me 👍🏻
@carlocab thoughts given the above?
I agree that we've fixed a lot of the issues with GCC on Linux, but we haven't fixed them all.
- Dependents of formulae built with Homebrew GCC on Linux still need to be built using Homebrew GCC. This is one of biggest factors that contributes to dependency creep wrt
gcc. - GCC's dependency tree is still much larger than is reasonable. This is bad not only for CI, when testing dependents, but also not great for users, whom we force the installation of a Homebrew GCC even when, based on our analytics data on user Linux distros, one isn't really needed for them.
But, more importantly, and there's no easy way to say this: Linux maintainers just aren't pulling their weight when it comes to maintaining Homebrew. This isn't an attack on any Linux maintainer in particular. You are all lovely people and I respect and admire the work you do for Homebrew.
However, the truth is that when I said "we've fixed" in the first sentence of this comment, that actually means I fixed those problems. I refuse to be left holding the bag when it comes to fixing problems that are unique to using Homebrew GCC on Linux. The motivation for this is partly selfish, but not completely: it's just not optimal for someone who isn't at all familiar with Linux and doesn't use Linux on a regular basis to be the one to handle these problems.
In the absence of a strategy addressing the issues above, or even code written down that reduces the CI burden of GCC's Linux dependency tree, I still think we should adopt this check to stop the bleeding, and revisit it when we're in a better position to handle this issue better.
Also, to reiterate/rephrase something I said in the post that started this PR:
Allowing Linux-only GCC dependencies just gives us bad incentives. The correct solution to a formula failing to build with our preferred GCC, when our preferred GCC is ridiculously old, is is to update our preferred GCC.
However, without this audit, it is too easy to reach for adding a Linux-only GCC dependency, and that just hurts us in the long run.
This audit is designed to keep us in check by forcing us to be responsible about keeping our bottling distribution up-to-date.
If we do that, and use the reasonably modern GCC that comes with it, then I seriously doubt we'll have problems with bottling formulae. If a formula fails to build with a recent-ish GCC that is still in wide circulation, then that's an upstream bug that should be fixed there rather than worked around on our side.
Ok, I've pushed a change that, based on local testing, should work on (1) macOS, and (2) Linux with HOMEBREW_SIMULATE_MACOS_ON_LINUX (which is what we have in our tap_syntax job).
If this works (CC @Rylan12), and no one's position has changed since they last commented, I intend to merge this at some point tomorrow.
Can I offer one suggestion to make this slightly more flexible? Could this audit be conditional on the version of the version of the gcc formula being more than 3 versions above version of GCC used in CI?
My reasoning for this is that it would give us the flexibility to use brewed GCC on Linux, if needed, for a limited window of time in the unlikely event that some very cutting edge formula needs a newer GCC than what is in CI before we've made the upgrade. If we use 3 versions as the max, that gives us 1 year after the release of a new Ubuntu LTS to switch before the GCC version difference would exceed 3.
I believe this still achieves the goal of limiting us from using brewed GCC to put off migrations, while affording some flexibility in the interim for a handful of formulae.
I also really thing we should merge https://github.com/Homebrew/homebrew-core/pull/107400 before doing this, as qt will be debottled if that is merged, and that will be excruciating for users to build from source. We got complaints about debottling some big formulae before and I suspect we will again soon.
I really wish we were willing to have an allowlist for this just to exempt qt, qt@5, llvm and texlive. Their dependents I'm less concerned about but making all users build any of those from source is going to be extremely unpleasant. I don't want to sound too pushy here but I feel like we can afford to do dependent testing for a few big formulae until the migration is done. glibc is about to ship and the automatic installation of glibc is also almost done as well. I think we are within days of completing the migration now!
Can I offer one suggestion to make this slightly more flexible? Could this audit be conditional on the version of the version of the
gccformula being more than 3 versions above version of GCC used in CI?
I'd rather we didn't and instead considered using a newer, non-LTS Ubuntu release once we start needing newer compilers and upstream refuses patches because our version is too old.
I also really thing we should merge Homebrew/homebrew-core#107400 before doing this, as
qtwill be debottled if that is merged, and that will be excruciating for users to build from source. We got complaints about debottling some big formulae before and I suspect we will again soon.
We are these being debottled rather than bottled in the PR merge process?
I think we are within days of completing the migration now!
The migration needs to align, at least somewhat, with a major or minor Homebrew/brew release.
Note: existing formulae with Linux-only GCC dependencies will not be affected by this in CI. This primarily stops us from adding new ones.
I'd rather we didn't and instead considered using a newer, non-LTS Ubuntu release once we start needing newer compilers and upstream refuses patches because our version is too old.
That's a much better solution. Hopefully it's never an issue, but all the changes we've made mean we should be much quicker at doing CI migration in the future.
Note: existing formulae with Linux-only GCC dependencies will not be affected by this in CI. This primarily stops us from adding new ones.
Sorry for not picking up on that earlier - given how close we are to doing the migration I'm hoping that this affects almost no formulae. There are only a couple of unbottled existing formulae that could potentially need a newer GCC right now, but they all have other unrelated problems that are blocking them from being bottled anyway. And the rate at which exiting formulae acquire a newer GCC dependency isn't that high. It's certainly reasonable to hold off on adding completely new formulae that need brewed GCC until the migration is done, given how close we are.