Why isn't musl-cross-make on par with musl.cc's version?

[firasuke]

Why isn't this version of musl-cross-make on par with musl.cc's version over at:

https://git.zv.io/toolchains/musl-cross-make

[richfelker]

Some mix of:

1. It looks like it has a lot of infrastructure that's diverged to do things outside the scope of mcm, like making a huge set of toolchains for all targets.

2. They haven't submitted patches/PRs for the things that are in-scope.

3. Some of the patches look like hacks, and nobody has submitted descriptions of the problems they're intended to solve looking for clean fixes.

[firasuke]

Fair enough.

Any plans on adding some of their commits, as they've improved support for several arches?

[richfelker]

Can you open individual issues for improvements you'd like upstreamed so there's a place to discuss them that's not just a catch-all, and so I know what there's demand for? If you do, feel free to link them all from a comment here for continuity.

[firasuke]

Will do.

Thanks for the much appreciated quick replies.

[zv-io]

Hi, those are mine. I'll try to explain the reasons why I'm not attempting to upstream changes.

First, let's consider Rich's repository to be a reference implementation. I make my fork available so that all toolchains on musl.cc can be reproduced from source (config files are provided and the repo can be checked out at a revision corresponding to those dates).

While I can't promise this is always true, it is my intent that the same inputs to Rich's version (if supported) will produce identical outputs using mine. That is, any invocation of make and having an appropriate config.mak file. If you find discrepancies, please email me (it's on the homepage).

If Rich's doesn't support it, it's "not officially supported". I'm just some guy on the Internet :3rd_place_medal: :) though the idea is that they complement each other.

It looks like it has a lot of infrastructure that's diverged to do things outside the scope of mcm, like making a huge set of toolchains for all targets.

This infrastructure resides in a single scripts/ directory. That could be pulled out into its own repository. I am, however, periodically syncing from upstream (last major change was "cowpatch").

There are additional scripts outside the repository (linked from the homepage) which are used for:

• Preparing a consistent build environment (pinned versions of all build-time dependencies)
• Packing toolchains into Docker images

Another area of divergence is that I've made it a requirement to support Fortran for all targets. The field of scientific computing in general doesn't have a turnkey toolchain solution, and musl.cc's existence is directly a result of my frustrations with some of my work engagements at the time.

They haven't submitted patches/PRs for the things that are in-scope.

I do not consider it "in-scope" to upstream hashes/patches for every possible GCC, binutils, or other component version that I'm personally seeking to support. Mine tends to lean closer to the bleeding edge. For example, I've had binutils 2.34 and GCC 10 support for a while, but it's not the default.

Would these be desirable upstream?

Some of the patches look like hacks, and nobody has submitted descriptions of the problems they're intended to solve looking for clean fixes.

Some of my patches are specific to host platforms Rich doesn't support, e.g. Windows.

Specifically speaking to the 32-bit RISC-V and GCC pre-9 OpenRISC support, yes, these too are hacks. Cross-compiling Windows-hosted Linux-targeting toolchains requires a few errors be ignored.

Some of the GCC 10 patches are also horrible hacks, but unfortunately I don't have the time to craft clean fixes for what are usually corner-case problems. I'd love for someone with time/interest to help with that. I volunteer my time to maintain this site; it's not an extension of my own project as it is for Rich. I'm not denying there might be other hacks, but please point me to them so they may be rectified.

It's for everyone else who, in a pinch, needs something that "just works". The folks who are on Page 17 of web searches, who have clicked through every Stack Exchange link, and who stumble upon this dated-looking webpage hoping it can help them build some code at 3am so they can generate plots and get some sleep. For the CI/CD folks looking to spot-check platform support.

For anyone else who might find it useful.

It's used extensively as a sanity check for various packages which want to test across 32- and 64- bit, big- and little- endian, RISC and CISC, etc. I try to maintain a consistent offering across all hosts, so whether you're using Windows, macOS, Linux, Solaris, etc. you'll always have binary-identical output.

I ensure that every copy of GCC built is able to bootstrap itself. Occasionally I build and test a few dozen open-source packages for each musl.cc-supported target.

It's probably fair to say we're all open to collaboration and improvements as time permits, or as the tide of demand for fixes or features rises.

[richfelker]

Another area of divergence is that I've made it a requirement to support Fortran for all targets.

I am very interested in upstreaming any patches needed for Fortran to work, altho not building it by default. Ada too.

For example, I've had binutils 2.34 and GCC 10 support for a while, but it's not the default.

Would these be desirable upstream?

For actual releases, yes, if you're committed to ensuring that patches don't get dropped/lost between versions.

Some of my patches are specific to host platforms Rich doesn't support, e.g. Windows.

Arbitrary host platforms able to handle the build mechanics (that means symlink support, etc.) should work for mcm. Only the target is Linux/musl-specific. If there are non-invasive fixes for Windows host, Apple host, etc. those are probably welcome.

[zv-io]

I am very interested in upstreaming any patches needed for Fortran to work, altho not building it by default. Ada too.

Ada is a work in progress. My rule of thumb, however, is that "if it doesn't work for all, it doesn't work for any" [of the musl.cc releases] and admittedly Ada's not a priority for me at this time (read: help is welcome). I'm aware of some patches for musl/Ada support but those are GCC version-specific and not working on most platforms last I checked. I'll eat my words if someone steps up with a counterexample.

For actual releases, yes, if you're committed to ensuring that patches don't get dropped/lost between versions.

I'm committed to not dropping patches from a given directory (e.g. patches/gcc-10.1.0/) but that's not a guarantee future patches won't be added if they're needed to support additional platforms or languages. The SH2 FDPIC patches for binutils 2.34 (ported from 2.33.1) was a bit of extra work and I haven't verified it's /perfect/ but maybe you can take a look.

Only the target is Linux/musl-specific. If there are non-invasive fixes for Windows host, Apple host, etc. those are probably welcome.

So there are a few things like: https://sourceforge.net/p/mingw-w64/mailman/message/36683679/ which probably shouldn't be upstreamed but are harmless to Linux targets.

I'm in the process of merging the old mingw-cross-make and my fork of that with the current musl-cross-make and cleaning all of it up. That's what my fork will look like and continue to evolve from, not straying too far from upstream. Not sure yet. I prioritize what's useful to me.

[wdlkmpx]

I came to know about musl.cc some time ago reading this very issue, and I adapted my scripts to use those toolchains. And I'm quite satisfied, thanks. It's sad when interesting stuff is somewhat hidden and it takes luck to find it.

I used to produce my own static builds using this repo's defaults, and seeing the mingw stuff working with static cross compilers following musl-cross-make rules and logic makes me think of endless possibilities.

But now I want to see if some weird stuff works, something like a static cross compiler for freebsd, solaris, openbsd, should I use this or that repo as the base to edit and make crazy changes, this fork or the fork of that fork, which one from the network graph. Hmm it gets confusing.