msys2
Several core packages aren't building successfully at the moment.
Describe the issue
I'm using an up-to-date clean MSYS2 environment to build the packages plus a few more added to it. I have tried building some of the main packages but be it for one thing or another they are broken at the moment. I couldn't find a way for them to build (logs attached).
For example:
- coreutils: error in build process, but I didn't get to the tests.
- pacman: fails 93 tests (+7 unexpected).
- fileutils: fails 17 tests.
- patch: fails 5 tests, xfails 2.
- grep: only passes 1 tests.
- openssh: fails some test(s).
- gettext: fails all of the tests.
Others while they seem to build just fine (bash, ncurses, libreadline) bring more problems to the mix when installed, there is an error already in bash' installation, but errors occur in other packages when they're dependencies. I.e., the package fails to build, or execute properly with the rebuild of one of them, but does work fine with those in the repos. Vim, for example, is finicky.
Steps to Reproduce the Problem
- Clean MSYS2 distribution.
- Have these packages installed (those are
pacman -Qqe): autotools base binutils diffutils filesystem gcc gdb git make man-db meson msys2-runtime ninja openssh patch pkgfile python vim - Build any of the packages mentioned above, possibly more.
- Successful builds with passed tests for those that have them. Ideally the whole core package distribution should be evaluated.
Additional Context: Operating System, Screenshots
- OS: Windows 10 Pro 21H2, current MSYS2, run in MinTTY.
- coreutils-makepkg.log
- pacman-makepkg.log
- findutils-makepkg.log
- patch-makepkg.log
- grep-makepkg.log
- openssh-makepkg.log
- gettext-makepkg.log
Perl is kind of a beast in itself, somewhere along the build process something goes wrong... around line 985 of the log you can see what goes on, but build finishes successfully.
However, several tests fail, but because they're output isn't taken into consideration (|| true) it gets to be packaged, who knows whether the build is performing as expected or not, I understand some things are supposed to fail or not even be evaluated because of the environment.
thanks, likely because we don't run tests when building our packages, using --nocheck.
I'll have a look at coreutils. edit: see #3078
I see! Is it useful if I report some other builds that don't succeed?
I could try to automate the process, using --nocheck if that's how you guys build, and report any of the base packages that for one reason or another refuse to build. If I see an easy way to fix them I'll let you know of course.
Some need to be checked, I have my eye in one or two that even though they build correctly, something isn't right because others experience breakages when they're rebuilt if the rebuilds are installed. I can't point my finger to which one(s) just yet, but I'll try to identify them.
I'm using the standard compilation flags too, to get as close as possible to the distributed binaries and not introduce any noise because of custom ones. LTO for example is notorious for breaking MSYS packages in my experience, even if they are end programs and not libraries (e.g., Vim), but switching to x86-64-v2 or v3 as the march may already make things misbehave.
I see! Is it useful if I report some other builds that don't succeed?
build errors definitely.
I could try to automate the process, using
--nocheckif that's how you guys build, and report any of the base packages that for one reason or another refuse to build. If I see an easy way to fix them I'll let you know of course.Some need to be checked, I have my eye in one or two that even though they build correctly, something isn't right because others experience breakages when they're rebuilt if the rebuilds are installed. I can't point my finger to which one(s) just yet, but I'll try to identify them.
I'm using the standard compilation flags too, to get as close as possible to the distributed binaries and not introduce any noise because of custom ones. LTO for example is notorious for breaking MSYS packages in my experience, even if they are end programs and not libraries (e.g., Vim), but switching to x86-64-v2 or v3 as the
marchmay already make things misbehave.
yeah, LTO with gcc on Windows isn't very stable. Not sure about v2/3, we target the same CPUs that Windows supports and x86-64-v2 is too new for that.
I don't think automating it would be as cut and dried as I thought afterwards 😅, I looked into building some dependency graphs that could show me the build order for the packages in base at least, but it's complex. I'll be rebuilding some packages nonetheless, so I'll report on any that can't be built.
heimdal for example seems to require flex and bison as makedepends, They're rather basic, I don't know why they weren't installed in my system though.
~~libiconv fails to build since the new toolchain update, something changed for the better I think, from what I could see there are several commas missing in the source:~~ EDIT: When commas are added there tests don't pass for BIG5-HKSCS:1999, so that's not the issue, perhaps it'd need to be written differently but missing commas aren't a typo.
You're pretty much right about LTO, I've had good results with it, but only when building using native toolchains, be it those in MSYS2 or 3rd parties like the one bundled with CLion nowadays or Equation.com's. Because binaries running under MSYS2/Cygwin need some special environmental requirements (to my mind, ASLR comes to mind being a no go) I'm guessing that's partly where the problem stems. Still, LTO-ing is not a priority to be honest, not to me anyway, I'm more interested in having a stable toolchain and tools that I can depend on
Looking into the Gettext errors, because none of the tests seemed to work and yet applications using it work as expected I realized it's because of coreutils mkdir, it doesn't seem to support setting permissions when creating a directory (understandable).
For example, mkdir -m 700 test would create the directory but through and error, and that's more or less the command used at the beginning of each tests in the gettext battery... so they all fail. Coreutils would need to be patched to at least ignore that, perhaps, being Windows and all I don't know what the alternative would be.
Same goes for symbolic links creation, ln -s right now doesn't create a symbolic link (by the way, I think symbolic link creation in Windows doesn't require a privileged account anymore, but I don't recall when it happened. At the momento the behavior of ln -s is to create a full copy of the directory in question, but then throw an error message, hence != 0 termination value so many tests across the board that check for proper symlinking fail.
It looks like something to tackle and test properly rather than an easy fix in this case, so I'm deferring it for later.
Another thing that caught my attention is building without executing tests, I saw Vim's were disabled because it seems to hang (confirmed), but some packages have the check() function enabled. I'm building them all without executing, I think it would be better to see which ones do fail, perhaps having a list to compare to and prevent regressions in the future, but again, tedious.
On another topic, I noticed that Windows 7 support is being phased out, perhaps it'd be a good time to re-enable CNG in libarchive? There's a notice in that package stating it's disabled because it breaks pacman in Vista. If I started looking into testing and ensuring no regressions, pacman would probably be the first package to deal with seeing as it's one of the pilars of MSYS2 (and so happy it is).
Nothing else for now 😊
Same goes for symbolic links creation,
ln -sright now doesn't create a symbolic link (by the way, I think symbolic link creation in Windows doesn't require a privileged account anymore, but I don't recall when it happened.
In Win 10/11 user can enabled unprivileged symlinks by enabling developer mode (it's disabled by the default).
aslr is indeed a nogo with the msys2 runtime (stated years ago with cygwin) since the runtime relies on a set baseadress. otherwise i agree packages should pass checks though since msys2 is a fork of cygwin we can reasonably assume them to work.
Hi, coreutils still fails to build. Apologies if you are already aware of this. Log is attached.
coreutils_build.log
