lmms
lmms copied to clipboard
LMMS
Update MinGW CI to 20.04
More often than not, the MinGW CI being outdated hampers development in the sense of having outdated libraries. A few PRs can't be merged just because of this outdated CI.
Though we are discussing on discarding MinGW entirely and moving to MSVC builds on windows, it'll still make sense to have updated mingw ci for the time being till we reach a consensus there.
Also @tobydox the PPA is severely out of date. Mind bumping all the packages to latest versions?
I don't think the MinGW builds need to target the oldest Ubuntu LTS like the Linux builds do, so we could probably target Ubuntu 24.04 instead.
GitHub build runners + vcpkg or MXE might be our best long term option for MinGW builds, and I've been experimenting a bit with them recently, but it may be a while before I have anything to show for it. If the PPA isn't too difficult to update, that would probably be our best short to mid term option.
I don't think the MinGW builds need to target the oldest Ubuntu LTS like the Linux builds do
Didn't know. Then using 24.04 seems good.
GitHub build runners + vcpkg or MXE might be our best long term option for MinGW builds
In that case, i vote for vcpkg as i am a bit familiar with it. Don't know about mxe tho, if mxe is better, then better to consider it.
Rebasing/updating all MinGW packages only makes sense for me if really required in the future. If you plan to migrate away from MinGW anyway, please understand that I don't want to invest the time then ;-) Otherwise I'm happy to support you with updating these packages.
Rebasing/updating all MinGW packages only makes sense for me if really required in the future.
Makes sense from a maintainer perspective. I didn't really thought of this.
you plan to migrate away from MinGW anyway
Tbh, we didn't really reach a consensus. Tresf is against this move when we discussed on discord. Though the votes are actually in favour of removing.
There's no major advantage to updating to an Ubuntu version later than 20.04 - our Linux builds are tied to that version for another year, so any features in newer versions aren't available to us yet anyway. The main advantage would simply be that we can go for longer without needing to upgrade again. I believe the Linux and MinGW Docker images share a common base, so using the same Ubuntu version may be simpler for now.
As for long-term MinGW use, we don't want to migrate away if we don't have to, but it may be a necessary move. The main obstacle to its use is the completeness of the C++ standard library implementation - a lot of threading-related parts are missing. We have a submodule (mingw-std-threads) that implements some of these missing parts, but it's not comprehensive. The choice right now is between sticking with increasingly outdated technology, or dropping support for MinGW, but the hope is that we find a way to avoid both of these options.
I guess the completeness of the C++ standard library heavily depends on the compiler version used. Ubuntu 24.04 comes with g++-mingw-w64 13.2 and mingw-w64 runtime 11.0.1 so I think it should have quite complete support, doesn't it? And since the oldest distribution supported by LMMS dictates the baseline anyway (e.g. Ubuntu 20.04 comes with GCC 9.3) you must not use any features that are not already supported by these compiler versions. Or do I fundamentally misunderstand something?
Previously, @sakertooth and @messmerd tried introducing some new features into the codebase and they were hindered by the lack of threading support and incomplete features in the MinGW standard library. They know the issue more than i do so let them tell.
@tobydox The MinGW cross-compiler we use does not implement the C++11 standard threading library (<thread>, <mutex>, <condition_variable>, etc.) even though we are using C++17 now. Those features were deliberately left out of MinGW's standard library, but apparently recent versions of MinGW-w64 include support for it via a Win32 port of pthreads. That's something we should look into. Currently we are forced to use this repo for a partial implementation of those standard library features, and it's not ideal.
@tobydox It looks like you had an Ubuntu 20.04 MinGW PPA already, but it just wasn't being used: http://ppa.launchpad.net/tobydox/mingw-w64/ubuntu/dists/focal/
So I rebuilt the MinGW Docker image using it and with some small adjustments, was able to get the CI builds working: #7259
MinGW-w64 include support for it via a Win32 port of pthreads. That's something we should look into.
I was looking into this, but couldn't confirm that I can compile using only MinGW (without mingw-std-threads) using their pthreads port. If this is in fact possible, then I don't see the need to drop MinGW. Though, I would think the MinGW version we are at right now would already have this port?
I don't think the MinGW builds need to target the oldest Ubuntu LTS like the Linux builds do, so we could probably target Ubuntu 24.04 instead.
Linux won't have to either, once #7252 is merged. Finally gets away from that lock-in mindset. We'll be able to choose whichever LTS target we want. (and no more Intel lock-in either).
our Linux builds are tied to that version for another year, so any features in newer versions aren't available to us yet anyway.
Happy to say we can make this decision ourselves once #7252 is merged.
MinGW-w64 include support for it via a Win32 port of pthreads. That's something we should look into.
I was looking into this, but couldn't confirm that I can compile using only MinGW (without mingw-std-threads) using their pthreads port. @sakertooth
I can confirm it, I did it a couple of days ago. Requires adjusting the headers a bit to drop all the ifology for the #include <mingw.xyz.h> from the 3rd party pthreads, but IMVHO it compiles and works as a charm. I'm more than happy to do the PR for that change, I already have the code (and it's quite simple as well) - I did it because mingw-std-threads breaks MinGW-on-Windows (MSYS2) builds due to incompatibilities in mingw.future.h implementation vs future from C++, so IMVHO dropping mingw-std-threads is the way to go (especially taking into account the spamming pragmas in the build process :D). A proof-of-concept PR is https://github.com/LMMS/lmms/pull/7283 .
If this is in fact possible, then I don't see the need to drop MinGW. Though, I would think the MinGW version we are at right now would already have this port?
MinGW has it for a long time already, if you choose POSIX model (see https://stackoverflow.com/questions/17242516/mingw-w64-threads-posix-vs-win32 for details). The files are provided by https://sourceforge.net/p/mingw-w64/mingw-w64/ci/master/tree/mingw-w64-libraries/winpthreads/ , and the exposure via C++ header is also kinda ancient. I know for sure that it worked for me out-of-the-box for at least two years now, and probably worked properly well before that.
The LMMS CI docker would have to be updated, though. ~AFAIK, Focal (20.04) doesn't provide the POSIX build of MinGW g++ lib directly, it's been avaiable from Jammy (22.04) though.~ I was somewhat wrong: actually, 20.04 and earlier have them but require update-alternatives, later Ubuntu versions allow just specifying the POSIX package directly in APT. I'm going through temporary fixes downstream ATM in the GH action to showcase what needs to be done to switch to POSIX model (which IMVHO should be the default in cross-platform build anyway). If you want, I can submit the relevant PR to LMMS CI repo as well as soon as I finish that.
BTW, if you guys are considering dropping first-class MinGW/MSYS2 support for any reason, I'm more than happy to help with it. IMVHO maintaining it doesn't really take that much effort vs regular Linux build, and I've been using MinGW/MSYS2 to build native (no dedicated MinGW support) Linux and Windows apps for long enough to see the value in it, even if just for myself.
@messmerd
I don't think the MinGW builds need to target the oldest Ubuntu LTS like the Linux builds do, so we could probably target Ubuntu 24.04 instead.
I second this. Without going for an update to a recent-ish LTS, it's a dep hell forever, due to having to conform to severely outdated libs. Mind me, even 24.04 has some delay with many of the packages, sometimes a year or two... With 20.04, it's about 5 years of delay vs bleeding edge deps ATM I'd say.
BTW last update of tobydox MinGW has been for Focal. Future-proof, another solution would be needed. For the time being, they can be used with 22.04/24.04 Ubuntu though.