v1.1.0-rc1 alr toolchain issues

[1ma]

Summary (by @mosteo):

• [x] Disable assistant (alr toolchain --disable-assistant) (#803)
• [x] Non-interactive selector (alr toolchain --select <release>) (#804)
• [x] gprbuild missing external
• [ ] Static linking

Original post follows.

unrelated commands such as alr clean spawn an interactive prompt to set up the toolchain

$ alr clean
Cloning into '/root/.config/alire/indexes/community/repo'...
remote: Enumerating objects: 5262, done.        
remote: Counting objects: 100% (430/430), done.        
remote: Compressing objects: 100% (253/253), done.        
remote: Total 5262 (delta 177), reused 341 (delta 117), pack-reused 4832        
Receiving objects: 100% (5262/5262), 848.20 KiB | 4.93 MiB/s, done.
Resolving deltas: 100% (3000/3000), done.
Welcome to the toolchain selection assistant
In this assistant you can set up the default toolchain to be used with any crate
that does not specify its own top-level dependency on a version of gnat or 
gprbuild.
If you choose "None", Alire will use whatever version is found in the 
environment.
Note: gnat is currently not configured. (alr will use the version found in the environment.)
Please select the gnat version for use with this configuration
  1. gnat_native=10.3.0-1
  2. None
  3. gnat_external=9.3.0 [Detected at /usr/bin/gnat]
  4. gnat_arm_elf=10.3.0-1
  5. gnat_riscv64_elf=10.3.0-1
Enter your choice index (first is default): 
> ERROR: Canceled.

Taken from https://gitlab.com/1ma/radatracer/-/jobs/1513532725#L23

I've been using alr clean for fetching the sources of dependencies without building them in the same step. When running alr clean Alire v1.1.0-rc1 spawns an interactive dialog asking the user to configure the toolchain, and since in this case it runs on a non-interactive context (a CI pipeline) it fails.

Maybe Alire 1.1.0 could start configured with the "None" toolchain and let the user alr toolchain --select one if he wants. I think that'd be a smoother user experience.

toolchain --select does not have a non-interactive mode

Related to the above problem, I don't seem able to fix the pipeline because I cannot select a toolchain non-interactively.

$ alr toolchain --select gnat_external
error: Specify the action to perform with the crate

There is also a --non-interactive (-n) flag, but what this does is it automatically picks the first option, not the one I might want.

Alire detects external FSF GNAT but not external GPRBuild

$ which gprbuild 
/usr/bin/gprbuild

$ ./bin/alr toolchain --select              
Welcome to the toolchain selection assistant

In this assistant you can set up the default toolchain to be used with any crate
that does not specify its own top-level dependency on a version of gnat or 
gprbuild.

If you choose "None", Alire will use whatever version is found in the 
environment.

ⓘ gnat is currently not configured. (alr will use the version found in the environment.)

Please select the gnat version for use with this configuration
  1. gnat_native=10.3.0-1
  2. None
  3. gnat_external=10.3.0 [Detected at /usr/bin/gnat]
  4. gnat_arm_elf=10.3.0-1
  5. gnat_riscv64_elf=10.3.0-1
Enter your choice index (first is default): 
> 3
ⓘ Selected tool version gnat_external=10.3.0

ⓘ gprbuild is currently not configured. (alr will use the version found in the environment.)

Please select the gprbuild version for use with this configuration
  1. gprbuild=21.0.0-1
  2. None
Enter your choice index (first is default): 
> 2
ⓘ Selected to rely on a user-provided binary.

gnat_native toolchain and gprbuild are linked against glibc

musl-based distros such as Alpine Linux cannot use the distributed native executables, as they are linked against glibc.

$ alr toolchain --install gnat_native
...

$ ldd .config/alire/cache/dependencies/gnat_native_10.3.0_8b7c5dcd/bin/gnatmake
        /lib64/ld-linux-x86-64.so.2 (0x7f266cd30000)
        libdl.so.2 => /lib64/ld-linux-x86-64.so.2 (0x7f266cd30000)
        libm.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f266cd30000)
        libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f266cd30000)
Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by .config/alire/cache/dependencies/gnat_native_10.3.0_8b7c5dcd/bin/gnatmake)
Error relocating .config/alire/cache/dependencies/gnat_native_10.3.0_8b7c5dcd/bin/gnatmake: fcntl64: symbol not found
Error relocating .config/alire/cache/dependencies/gnat_native_10.3.0_8b7c5dcd/bin/gnatmake: __sched_cpualloc: symbol not found
Error relocating .config/alire/cache/dependencies/gnat_native_10.3.0_8b7c5dcd/bin/gnatmake: __sched_cpufree: symbol not found
Error relocating .config/alire/cache/dependencies/gnat_native_10.3.0_8b7c5dcd/bin/gnatmake: _r_debug: symbol not found

$ apk info musl
musl-1.2.2-r3 description:
the musl c library (libc) implementation

musl-1.2.2-r3 webpage:
https://musl.libc.org/

musl-1.2.2-r3 installed size:
608 KiB

These executables could be linked statically, or separate releases linked against musl could be published.

[mosteo]

Thanks for the detailed report, Marcel.

Although I see your point, alr clean requires a gprclean, so the compiler should be configured to run it. In practice, the assistant is being triggered by any command that requires an alire.toml. (Also, why not alr update instead of clean?)

@Fabien-Chouteau , I'm inclined to disabling the autorun of the assistant, and point to it in the error message when gprbuild is not found.

Independently of that, being able to use alr toolchain --select [release] for non-interactive selection looks like a good idea to me.

The undetected gprbuild is a matter of adding the missing external; will do shortly.

[Fabien-Chouteau]

@Fabien-Chouteau , I'm inclined to disabling the autorun of the assistant, and point to it in the error message when gprbuild is not found.

I don't know, the assistant is great for beginner and will save a lot of troubles. It should show up only once, although it could still be an issue for continuous integration runs.

[Fabien-Chouteau]

@1ma

gnat_native toolchain and gprbuild are linked against glibc

musl-based distros such as Alpine Linux cannot use the distributed native executables, as they are linked against glibc. [...] These executables could be linked statically, or separate releases linked against musl could be published.

Do you have the same problem with GNAT Community 2021? Do you know how to build gcc in a compatible way?

[mosteo]

It should show up only once, although it could still be an issue for continuous integration runs.

I faced the same problem for our tests, but I knew how to bypass it using alr config. Perhaps we can simple have an alr toolchain --disable-assistant. That, plus being able to select compilers with the alr toolchain --select <release>, would allow to configure alire for headless tests without needing to know about internals.

[Fabien-Chouteau]

Will the assistant take a good default toolchain when in non-interactive mode?

[mosteo]

Will the assistant take a good default toolchain when in non-interactive mode?

The default option ATM is the newest native compiler from the index.

[Fabien-Chouteau]

So everything should be fine in non-interactive mode.

[mosteo]

I think so. If you run alr toolchain --select non-interactive it will install the packaged compiler. If you don't want that then run alr toolchain --select <whatever> (TBD) to select another, or alr toolchain --disable-assistant (also TBD) to stay unconfigured.

[1ma]

Do you have the same problem with GNAT Community 2021?

Yes.

d78276a1db7a:/# cat /etc/alpine-release 
3.14.1

d78276a1db7a:/# ./gnat-2021-20210519-x86_64-linux-bin 
sh: ./gnat-2021-20210519-x86_64-linux-bin: not found

d78276a1db7a:/# ldd gnat-2021-20210519-x86_64-linux-bin
	/lib64/ld-linux-x86-64.so.2 (0x7fe1a889b000)
	libutil.so.1 => /lib64/ld-linux-x86-64.so.2 (0x7fe1a889b000)
Error loading shared library libfontconfig.so.1: No such file or directory (needed by gnat-2021-20210519-x86_64-linux-bin)
Error loading shared library libfreetype.so.6: No such file or directory (needed by gnat-2021-20210519-x86_64-linux-bin)
Error loading shared library libdbus-1.so.3: No such file or directory (needed by gnat-2021-20210519-x86_64-linux-bin)
...

Do you know how to build gcc in a compatible way?

Not readily, but I'm looking into it. Rich Felker (musl author) has a project for building musl-targeting cross-compilers: https://github.com/richfelker/musl-cross-make Ada is not officially supported but with a couple patches that I fished from Alpine Linux I got it to build GNAT 9.4.0 and it seems to work.

I've also realized that providing a single static x86_64 toolchain for Linux is not enough. Even if the compiler itself can run on musl (because it is static) the compiled programs will still target x86_64-linux-gnu. The only way would be to have a separate toolchain targeting x86_64-linux-musl. Maybe this is too much work at the moment, ~~but I think that if you want to provide toolchains for other architectures in the future (say, ARM or RISC-V) you'll have to do something like that anyway~~ actually you already do :sweat_smile:

[1ma]

The following Dockerfile shows how to build a static GCC 9.4.0¹ targeting x86_64-linux-musl. I've been able to build XmlAda, GPRBuild, GNATColl, Alire and my own projects with this toolchain.

I've also taken a cursory look at GNAT-FSF-builds but I can't figure out how it ticks. If you want to build this image and test it yourselves: $ DOCKER_BUILDKIT=1 docker build -t gcc:alpine -f Dockerfile . then $ docker run --rm -it gcc:alpine sh -l.

To install other packages inside the container you use apk add, e.g. apk add nano git make

To extract the toolchain from a running container into your host get its name with docker ps then docker cp nice_ishizaka:/usr/local/gcc .

The patching is admittedly a bit clunky, I'll try to get it merged into musl-cross-make.

# syntax = docker/dockerfile:1.3-labs
FROM alpine:3.12

WORKDIR /tmp

ADD https://github.com/richfelker/musl-cross-make/archive/master.tar.gz .

RUN tar zxf master.tar.gz \
 && apk add --no-cache \
    build-base \
    gcc-gnat

RUN <<'EOF' cat > /tmp/musl-cross-make-master/config.mak
TARGET = x86_64-linux-musl

GCC_VER = 9.4.0

COMMON_CONFIG += CC="gcc -static --static" CXX="g++ -static --static"
COMMON_CONFIG += CFLAGS="-g0 -Os" CXXFLAGS="-g0 -Os" LDFLAGS="-s"

BINUTILS_CONFIG += --program-prefix=

GCC_CONFIG += --enable-languages=ada,c,c++ --program-prefix=
EOF

RUN <<'EOF' cat > /tmp/musl-cross-make-master/patches/gcc-9.4.0/0020-ada-compat.diff
diff --git a/gcc/ada/Makefile.rtl b/gcc/ada/Makefile.rtl
index 775ab9857..0b7232a83 100644
--- a/gcc/ada/Makefile.rtl
+++ b/gcc/ada/Makefile.rtl
@@ -2394,7 +2394,7 @@ ifeq ($(strip $(filter-out %x86_64 linux%,$(target_cpu) $(target_os))),)
   s-taspri.ads<libgnarl/s-taspri__posix.ads \
   g-sercom.adb<libgnat/g-sercom__linux.adb \
   $(TRASYM_DWARF_UNIX_PAIRS) \
-  s-tsmona.adb<libgnat/s-tsmona__linux.adb \
+  s-tsmona.adb<libgnat/s-tsmona.adb \
   $(ATOMICS_TARGET_PAIRS) \
   $(X86_64_TARGET_PAIRS) \
   system.ads<libgnat/system-linux-x86.ads
diff --git a/gcc/ada/libgnarl/s-osinte__linux.ads b/gcc/ada/libgnarl/s-osinte__linux.ads
index 9d5cd0525..865de4dfb 100644
--- a/gcc/ada/libgnarl/s-osinte__linux.ads
+++ b/gcc/ada/libgnarl/s-osinte__linux.ads
@@ -398,12 +398,6 @@ package System.OS_Interface is
    PTHREAD_RWLOCK_PREFER_WRITER_NP              : constant := 1;
    PTHREAD_RWLOCK_PREFER_WRITER_NONRECURSIVE_NP : constant := 2;

-   function pthread_rwlockattr_setkind_np
-     (attr : access pthread_rwlockattr_t;
-      pref : int) return int;
-   pragma Import
-     (C, pthread_rwlockattr_setkind_np, "pthread_rwlockattr_setkind_np");
-
    function pthread_rwlock_init
      (mutex : access pthread_rwlock_t;
       attr  : access pthread_rwlockattr_t) return int;
@@ -465,11 +459,6 @@ package System.OS_Interface is
       protocol : int) return int;
    pragma Import (C, pthread_mutexattr_setprotocol);

-   function pthread_mutexattr_setprioceiling
-     (attr        : access pthread_mutexattr_t;
-      prioceiling : int) return int;
-   pragma Import (C, pthread_mutexattr_setprioceiling);
-
    type struct_sched_param is record
       sched_priority : int;  --  scheduling priority
    end record;
diff --git a/gcc/ada/libgnarl/s-taprop__linux.adb b/gcc/ada/libgnarl/s-taprop__linux.adb
index c45559e5b..f5b41c943 100644
--- a/gcc/ada/libgnarl/s-taprop__linux.adb
+++ b/gcc/ada/libgnarl/s-taprop__linux.adb
@@ -349,6 +349,8 @@ package body System.Task_Primitives.Operations is
    ----------------

    function Init_Mutex (L : RTS_Lock_Ptr; Prio : Any_Priority) return C.int is
+      pragma Unreferenced (Prio);
+
       Mutex_Attr : aliased pthread_mutexattr_t;
       Result, Result_2 : C.int;

@@ -365,10 +367,6 @@ package body System.Task_Primitives.Operations is
            (Mutex_Attr'Access, PTHREAD_PRIO_PROTECT);
          pragma Assert (Result = 0);

-         Result := pthread_mutexattr_setprioceiling
-           (Mutex_Attr'Access, Prio_To_Linux_Prio (Prio));
-         pragma Assert (Result = 0);
-
       elsif Locking_Policy = 'I' then
          Result := pthread_mutexattr_setprotocol
            (Mutex_Attr'Access, PTHREAD_PRIO_INHERIT);
@@ -409,11 +407,6 @@ package body System.Task_Primitives.Operations is
             Result := pthread_rwlockattr_init (RWlock_Attr'Access);
             pragma Assert (Result = 0);

-            Result := pthread_rwlockattr_setkind_np
-              (RWlock_Attr'Access,
-               PTHREAD_RWLOCK_PREFER_WRITER_NONRECURSIVE_NP);
-            pragma Assert (Result = 0);
-
             Result := pthread_rwlock_init (L.RW'Access, RWlock_Attr'Access);

             pragma Assert (Result in 0 | ENOMEM);
diff --git a/gcc/system.h b/gcc/system.h
index d04f8fd33..5505eeb74 100644
--- a/gcc/system.h
+++ b/gcc/system.h
@@ -864,7 +864,7 @@ extern void fancy_abort (const char *, int, const char *)
 #undef calloc
 #undef strdup
 #undef strndup
- #pragma GCC poison calloc strdup strndup
+ #pragma GCC poison strdup strndup
 #endif

 #if !defined(FLEX_SCANNER) && !defined(YYBISON)
EOF

RUN make -C musl-cross-make-master -j$(nproc) \
 && make -C musl-cross-make-master install


FROM 1maa/alpine:3.14

ENV PATH=/usr/local/gcc/bin:$PATH

COPY --from=0 /tmp/musl-cross-make-master/output /usr/local/gcc

1. 9.4.0 is the most recent version available, though there are open PRs for adding GCC 10.3.0 and 11.1.0

[Fabien-Chouteau]

So we need a special build of GCC for musl/Alpine. This can probably be done in GNAT-FSF-builds but I don't have time for this right now. Also we would first need Alpine distrib support in Alire first.

[1ma]

Alright. For what it's worth, for the last few months I've been building and using Alire 1.0.0 and 1.0.1 on Alpine Linux without a hitch.

[AJ-Ianozi]

I decided to spin up an alpine VM to mess around with this today. I successfully pulled in the current version of gnat via apk add gcc-gnat and apk add musl-dev. Then I built and installed gprbuild manually, then built alire.

alr runs fine except the native toolchain it brings in of course doesn't work:

localhost:~/hello_1.0.2_5715870b$ /home/aj/.cache/alire/toolchains/gprbuild_22.0.1_24dfc1b5/bin/gprbuild --version
-ash: /home/aj/.cache/alire/toolchains/gprbuild_22.0.1_24dfc1b5/bin/gprbuild: not found
localhost:~/hello_1.0.2_5715870b$ file /usr/local/bin/gprbuild
/usr/local/bin/gprbuild: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-x86_64.so.1, with debug_info, not stripped
localhost:~/hello_1.0.2_5715870b$ file /home/aj/.cache/alire/toolchains/gprbuild_22.0.1_24dfc1b5/bin/gprbuild
/home/aj/.cache/alire/toolchains/gprbuild_22.0.1_24dfc1b5/bin/gprbuild: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, with debug_info, not stripped

I know right now we haveos that can be linux, windows, macos, bsd(?), etc in alire.

While most distros use glibc, it looks like quite a few are using musl now. ~~One easy solution could be to default to "linux" as the os and then offer "alpine", "talos", etc. which would be the most backwards-compatible.~~

~~As we add support for alpine, we could have something like [origin."case(os)".alpine."case(host-arch)".x86-64] in there.~~

~~This would means opening up several more options for host_os, which looks like it's something hard-coded into alire right now, so I'm not sure if that's worth doing or not.~~ The more I think about it, the more I think the os thing is a bad idea.

An alternative is to add something like libc_variant (that defaults to being glibc?) or similar that's passed to alire or detected on build. Maybe then gprbuild's toml will look something like:

[origin."case(os)".linux."case(libc_variant)".glibc."case(host-arch)".x86-64]
url = "https://github.com/alire-project/GNAT-FSF-builds/releases/download/gprbuild-22.0.0-1/gprbuild-x86_64-linux-22.0.0-1.tar.gz"
hashes = ["sha256:24dfc1b54655edd4f85589e7e7dbd0bee24d087f25d5b0f13d3224fe7acf85b8"]

[origin."case(os)".linux."case(libc_variant)".musl."case(host-arch)".x86-64]
url = "https://github.com/alire-project/GNAT-FSF-builds/releases/download/gprbuild-22.0.0-1/gprbuild-x86_64-linux-alpine-22.0.0-1.tar.gz"
hashes = ["sha256:hashgoeshere"]

# Would this help for backwards-compatibility?  We default to glibc.

[origin."case(os)".linux."case(host-arch)".x86-64]
url = "https://github.com/alire-project/GNAT-FSF-builds/releases/download/gprbuild-22.0.0-1/gprbuild-x86_64-linux-22.0.0-1.tar.gz"
hashes = ["sha256:24dfc1b54655edd4f85589e7e7dbd0bee24d087f25d5b0f13d3224fe7acf85b8"]

What do you think? As a bonus, a libc_variant could someday be msvc or something on windows.

[mosteo]

The main issue here is fixed. For the musl ideas I advise to open a separate issue.